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PREFACE - AN OVERVIEW OF PART II OF THIS REPORT 



Part I of this report (Technical Report. No. 72-117) was subtitled "A 
General Discussion", and presented the pragmatic answers we decided upon to 
the "philosophical" questions involved in systematically attempting to ob- 
tain student opinion about the teaching to which University students are 
exposed. Part II is a logical continuation of Part I, in that it presents 
the detailed procedures that were developed and are presently used within the 
Department of Physics & Astronomy. (These procedures have been given the 
acronym PATS — for Physics & Astronomy Teaching Survey.) 

Structurally Part II is written as a detailed administrative and opera- 
ting manual L'jt PATS. This is divided into three separate parts; it is 
likely that these three parts will be under the supervision of three separate 
individuals, and to a major extent the three parts are independent and can be 
read separately. Chapter 3* is directed to the overall administration of the 
questionnaires, up to the stage of converting the responses into "Digitek" 
computer readable IBM cards. Chapter 3 also discusses certain aspects of the 
preparation for the computer runs of the programs DATAREAD, INITPRT and 
AGGFORM; and finally, it discusses the distribution process for the results 
of the computer analysis and sundry other administrative matters. Chapter 4 
discusses the first stage of the computer analysis of the Digitek cards, which 
uses the computer programs DATAREAD and INITPRT (initial print). Chapter 5 
clis cusses the computer program AGGFORM (aggregate formation) which implements 
the philosophy given in Part I. Chapter 5 is intended for the person who will 
interface with the computer and is really an AGGFORM execution manual. Since this 
person must be given instructions by the person in charge of overall admini- 
stration, therefore such instructions are discussed in Chapter 3. However, 
for simplicity a brief review of these relevant ideas are given in Chapter 5 
so that Chapter 5 remains self-contained. 

There are also numerous appendices. 

For the reader* s information, we repeat in the following pages the PRE- 
FACE, from PART I of this report, as well as the updated and amplified 
ACKNOWLEDGEMENTS. 



We continue the Chapter numbers and pagination from Part I. 
Appendices in Part II are given letter identifiers. 



PRI;FACIi: - AN OVERVIEW OF THIS REPORT 



Ihis report is addressed to the problems of how systematically to ob- 
tain student opinion about the teaching to which such students are exposed 
ill a University setting. Nowhere do we consider the very important matter 
of how to interprec such "uneducated" opinion.'^ We call sudi opinion "student 
evaluation of teaching" without any implied judgment concerning the validity 
of this student evaluation. 

Tills report specifically describes the procedures presently used by 
the Department of Physics and Astronomy. (These procedures have been given 
the acronym PATS - for Physics and Astronomy Teaching Survey.) In order to 
develop PATS, many general problems had to be addressed and decisions made. 
In particular it was found that "purely technical" questions x^e re inescap-^ 
ably intertwined with "philosophical" questions, and many such philosophical 
questions had to be asked and answered before one could fully implement PATS. 
Such questions must always arise in any attempt to obtain student evaluation 
of teaching. The danger is that certain administrative type decisions on 
"technical questions" may be made without full realization that they have 
philosophical import. 

It follows that this report must address itself both to general ques- 
tions relating to all such student evaluation procedures, and to the specific 
procedures used in PATS. We hence decided to write this report in two sepa- 
rate parts, separately bound. Part I deals primarily with general questions, 
and should be of interest to anyone who is attempting to set up a student 
evaluation procedure. Part II is primarily an operating manual for PATS in 
its present form. It would be of interest to someone who is charged with 
the actual implementation of such a similar procedure, since he must deal 
with all the technical questions. But our hope is that all general type 
questions are raised and discussed in Part I. These are the types of ques- 
tion which would need to be considered, for instance, by a faculty committee 
overseeing the implementation of such a procedure. 

The two parts of this report are so written that each may be read on 
its o\m. Hence it has been necessary to duplicate certain material. Struc- 
turally Part II is the continuation of Part I, and the two parts together 
form the complete report. 

We have organized the report as follows: Part I consists of Chapter 1 
and 2 and related appendices. Chapter 1 provides a general overview of 
PATS, from the nature of the questionnaires to a brief description of the 
way in which the responses are processed and the results made public. A 
summary of the entire process is given in the final section of this chapter. 
Chapter 2 considers some of the problems and decisions involved in such a 
project, such as how one goes about combining the results of different 



But is it any more uneducated then most other forms of teaching evalua- 
tion? 



This preface aL<^o appears in Part I. 
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sections. This chapter explains how these questions were resolved for PATS, 
some of them being decisions of taste or convenience, with little compelling 
force behind them. Since most of the ''philosophical" problems are connected 
with combining the results of different sections, this chapter is principally 
a discussion of procedures used in the computer program AGGFORM.*'^ A number 
of concepts and definitions are introduced in this chapter. 

Part II of the report consists basically of "how to" manuals for the 
actual execution of the various aspects of PATS. Chapter 3 discusses over- 
all administration and supervision. Chapter 4 treats the preparation of 
data for and execution of DATAREAD and INITPRT. Finally, Qiapter 5 simi- 
larly discusses the preparation of data for, and execution of, AGGFORM. 
(There are also numerous appendices.) We have attempted to make these three 
chapters and related appendices each complete enough in itself so that, for 
example, different individuals could be responsible for different phases of 
the survey, and each would only have to read the one chapter relevant to his 
phase. Since various concepts and definitions are introduced in this report 
as needed, and a particular chapter may assume that the reader has already 
learned a particular term, such definitions have been collected in a Glossary 
for the convenience of those using, say just one chapter of the report. In 
some cases the Glossary contains illustrative examples not found in the main 
text. This Glossary is attached to the report as a^ Appendix which 
appears in Part II.* 



While PATS is an "official" survey procedure as presently implemented 
by the Department of Physics and Astronomy, it must be emphasized that this 
report is the work of four individuals; they were all deeply involved in 
setting up PATS, and hence they (particularly Dr. Claude K^cser) were forced 
to make many "philosophical" decisions as they arose during the task of tech- 
nical development. 'This report discusses these decisions. It follows that 
all such discussion should definitely not be construed as representing the 
"vifew of the Department of Physics and Astronomy," but solely as representing 
the views of the four individual authors. 



PATS is an ongoing operation. Hence it is continually being modified, 
updated, and hopefully improved. At any given stage, some minor errors 
exist. It has not seemed worthwhile to attempt to correct all of them in 
this present report. Thus questionnaire 3 contains some unfortunate "typo- 
graphical" errors, which will be corrected in the next run. But such 
errors hopefully should not detract from the overall value of this report. 
For similar reasons, we include unedited internal department memos. 



We consistently use the term "section" to denote the smallest basic unit 
of teaching exposure to one instructor, whether this refers to lecture, recita- 
tion or lab; faculty or graduate T/A; and whether one section of a many section 
course, or one distinct course. 

DATAREAD, INITPRT, and AGGFORM are the names of three separate electronic 
computer programs that are used by PATS at various stages. 

^As actually written. Part II does not contain such a complete glossary. A 
partial glossary appears in section 5.2> p. 5.1, and other concepts are 

discussed in Ch. 3« 
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CHAPTER 3> DETAILED ADMINISTRATION OF PATS 



3*1 Outline of Chapter 

In this chapter we provide, in a "chronological" sequence, the detailed 
instructions for the Initial data collection stages of administering PATS. 
This includes preparation of some of the data and instructions 'for the various 
computer programs. Fuller details for program running are provided in the 
computer program descriptions in the next two chapters. 

Naturally, we leave some details to the imagination* The present pur- 
pose is to provide enougii information so that PATS can be administered. But 
we do not intend to be encyclopedic. 

Terminology . A "section" refers to the basic unit being evaluated, 
whether lecture, lab, or recitation; whether faculty or teaching assistant. 
It generaxiy has one individual who is being evaluated with that section 
with that questionnaire. (Other terms are defined in Section 5.2, p. 5.1). 

We present in Table 3.1 (already given in Part I as Table 2) a flow chart 
of the "step-by-step" operation of PATS, As stated in the note, this chart 
is a "logical" one, and the "responsibility" for the different parts and 
even the sequence in which they are performed does not follow this order. 

3.2 The Questionnaires 

At present PATS uses 7 different questionnaires, to cover 9 different 
teaching experiences. These questionnaires are given in full in Appendix Al, 
with the list of courses assigned to each questionnaire given in Appendix A2. 
Appendix A3 gives a list of "corrections" to the questionnaires for future 
use when they are reprinted. Appendix A4 repeats Table 1 of Part I, and 
gives a brief description of each type of questionnaire.* 

3.3 Preparation of Packets for Distribution to Instructors 

For each section, a large manila envelope is prepared [some large sections 
require a box] with the following information written on the outside: 

name of instructor (T/A*s name if appropriate) 
course number 
section uTunber 

lecture /lab /recitation, as appropriate 
enrollment ^as tip to date as possible) 
code ntmibers 

The code number is explained further below, in section 3.4. The envelopes 
are also "rubber-stamped" with (see p. 3.3): 



^Naturally one can change any of the questions on the questionnaires at any 
time. The present programs are written to accomodate up to 80 questions, 
each with up to five different choices of answers, plus "abstain". Ex'^eed- 
ing such limits would require programming changes. 



TABLE 3,1 STEP-BY-STEP OPERATION OF PATS 

A. Administering the Questionnaires 

1. Packets of questionnaires and st«3iidard answer sheets giuen to each 
instructor. 

2. Each instructor informed of the 9--digit code number which identifies 
his section. 

3. Students fill out answer sheets in class, entering code number on 
these sheets. 

4. Filled-out answer sheets turned in to departmental office. 

B. Preparing: and Running DATAREAD and INITPRT 

1; Answer sheets converted to IBM cards by Dlgitek processor. 

2. Header cards prepared for each section (giving instructors' names, 
etc.) and placed immediately before Digitek cards for that section. 

3. Control cards prepared for DATAREAD. 

4. Data processed by DATAREAD. 

5. Corrections made to data as needed, DATAREAD rerun if necessary. 

6. INIPRT takes output of DATAREAD and prepares printout for each section. 

7. Printout checked for errors, corrections made to data if necessary. 

C. Preparing and Running AGGFORM 

1. It is decided what aggregates will be formed.. 

2. Descriptive title and a unique l-to-4 digit number assigned to each 
aggregate. 

3. Number of copies of printout needed for each aggregate is determined. 

4. Control cards prepared for AGGFORM^ 

5. AGGFORM reads tape produced by DATArEAD, forms and aggregates and 
prints results as directed by control cards. 

6. AGGFORM output checked for errors, particularly in control cards. 
Program re-run as necessary. 

7. The desired number of printed copies of the results for each 
Siggregate and fo-^ each single section are produced. 

8. Each instructor is given a copy of the results for his section and 
for all aggregates in his appropriate heirarchy. 

9. All single section and all aggregate results are put on public display. 



NOTE: This table shows a "logical" flow chart of operations. In fact many 

of the steps can and should be done in a different temporal order from 
that shown here. 



3.3 



! 



ENROLLMENT Please enter number of st udents actuall y 

participating in course 



Also describe aiiy special situations 

Header cards (see section 3.6 below) may be provisionally prepared at 
this time, or this may be delayed (see section 3.6). The choice is based 
primarily on the time schedule. If output is wanted as soon as possible, 
header cards should be prepared in advance, with the expectation that some 
changes will need to be made. At present questionnaires are issued about 
three weeks before the end of the term (partly to avoid last minute pres- 
sures, and partly to ensure polling labs which sometimes end, except for 
make-ups, before the end of the semester). Instructors then have a certain 
latitude in deciding when to issue them. Since as a policy matter output 
is not made public until after the end of the exam period, then there is a 
reasonable time available for processing. However, Digitek processing tends 
to be delayed, since it competes with hourly and final examination grading 
by Digitek throughout the University, including UMBC. 

The envelopes are then filled with questionnaires, answer sheets 
(standard Answer Sheets-TypeA-produced by Optical Scanning Corporation, 
and available from the Student Ur-i.on Bookstore [see Appendix B.l]) sharpened 
//2 pencils with erasers, and instructions to the instructor (see Appendix B.2). 
These packets are distributed, and all the same material is subsequently re- 
turned with hopefully the filled in answer sheets separated from those that 
are not used. 

Note that an effort is made to determine 'the number of students "par- 
ticipating" in the course (i.e. excluding those who have given up and are 
"taking an F"), the number of students who while present deliberately ab- 
stained from filling in the answer sheets, and finally the number of full 
respondents.* [See introduction, instructions, and question 1 on each ques- 
; tionnaire. Appendix A.l; also instructions to the instructor. Appendix B.2]. 

These efforts do not succeed completely. 

[ 3.4 Code Numbers 

i 

The standard answer sheet has a space for a 9 digit "student number", 
f It also has several other spaces available. The Digitek process (as present- 

) ly wired) can read this 9 digit number plus at most two more digits. We 

first tried using all 11 digits available, but found that students entered 
the numbers incorrectly. At present, we therefore use only the 9 digit 
"student number" for a code number that completely identifies the section. 
' All computer programs assume that a section is identified fully by a 9 

digit code number. However, the format within which the identification 
; r is "packed" within the 9 digits is fully variable. 

At present the format is as shown in Table 3.2, p. 3.4. 

*A11 of these except the number "participating" are determined internally by 
the various programs. There is thus no need to count the number of returned 
answer sheets. 
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TABLE 3.2 CODE NUMBER FORMAT 



digits 1, 2, 3: 
digit 4: 



digits 5, 6, 7: 
digit 8: 
digit 9: 



3 digit course number 

course designation: 

0 if "regular" physics 

1 if "Honors" physics 
2-4 future physics use 
5 if regular astronomy 

^ 6-9 future astronomy use 

r 3 digit section number 

1 (excluding the first zero) 

f questionnaire number 
I (presently 1-7) 

^1, 2, 3, 4 "professors" being 
evaluated, arranged alphabetically 
if needed, all within same course 
and section (1 if only 1 prof.) 

5, 6, 7, 8, 9 teaching assis- 
tants being evaluated, arranged 
alphabetically if needed (5 if 
only one teaching assistant) 



column: 


12 3 


A 


5 6 7 


8 


9 


content: 


course number 


Phys/Astron 
Hons 


section number 


quest. 

it 


prof/ ; 

/TA ; 
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The purpose of digit 9 is to allow for the two uses of questionnaires 
6 and 7 [independent lab courses with both "professors*' and "teaching 
assistants" — though whether these roles are filled by actual faculty or 
teaching assistants is iinniaterial] ; also to cover the cases when more 
than one instructor is associated with one section (including team-teaching). 

Digits 4 and 9 are read by the various computer programs, but are 
not used by them in an active way. Digit 8 (questionnaire number) is 
"tested" by the program. 

One could use a longer code number, and even have a special answer 
sheet printed and a special Digitek plug board wired. PATS prefers to use 
currently available materials. 

While generally it is best to follow the format given in Table 3.2 
uniformly, it is permissible to give "fraudulent" code numbers to sections 
for special purposes, e.g. 111222333. DATAREAD will provide a warning of the 
"illegal" code number when it comes to that section, but otherwise proceed 
normally. For instance, if one actually wants to subdivide the respdrrsW 
for one "official" registrar's section by some other fact (e.g. day of 
week) one could use fraudulent sections numbers, or perhi^ps fraudulent codes 
for digit 9. 

Further, (see also section 3.6 below), while generally the code num- 
ber given to the students will be the same as that later used on the header 
card, it is permissible for these to differ. DATAREAD will use the num- 
ber on the header card to determine course, section, etc. This freedom 
is useful when an "error" is detected in the number placed on the packet. 
Naturally no two sections may have the same header code number, but PATS 
can accept identical code nunibers on student answer sheets in different 
sections, which may occur by errpr. 

AGGFORM keeps careful count of numbers of sections and numbers of 
students. Hence, if a single section contains students registered for 2 
different courses (e.g. Physics 140 and 209; now 429 and 621) only one 
packet should be prepared — otherwise when attempting to combine the re- 
sponses later, two "sections" will be recorded. [This is a defect of the 
present AGGFORM — it would be desirable to alter the program to allow for 
such "divided" single sections.] Similarly, truly separate sections should 
not be coitbined in one packet with one code number, since this will cause 
errors in all subsequent aggregating. 

3.5 Digitek Processing 

When the packets are returned, all material other than the used answer 
sheets are removed. [Both totally filled in sheets, and positive abstain- 
ers* sheets are kept.] A cursory spot check to see that the written 



description on each answer sheet agrees with that on the outside of the 
packet is perhaps in order at this time. But there is no need to fill in 
missing code numbers, or correct those entered incorrectly. 

If >by error, responses from different sections have been mixed together 
in one envelope so that they cannot be identified by section, it is necessary 
to divide them at random into the true (or new) section envelopes, and put 
a suitable comment on the envelopes and header cards (see below). This 
is because AGGFORM keeps careful count of numbers of sections involved in 
aggregating. 

For Digitek processing, packets corresponding to questionnaires having 
up to 40 questions must be separated from those having 41-80 questions, 
but no other sorting or arranging is needed. Answer sheet packets ar-e -taken 
to the Digitek operator at the Computer Science Center [presently Ms. Linda 
Chase , in the keypunch room, x2941 — access through the Dispatcher's 
Room at north entrance], together with an interdepartmental billing author- 
ization. [The cost is presently 40 per answer sheet, and will be raised to 
100 as of July 1, 1973.] 

The Digitek operator must be given the following information and in- 
structions: 

1. The "Student" number is 9 digit. No other "identification" 
columns are used. 

2. After the machine selects out sheets with n£ "student" number 
marked, or with badly entered numbers, the$e sheets should be 
processed. Such sheets should be kept separate (turned face 
down) and such cards should be kept separate (by rubber band) — 
within the appropriate packet. There is no need to have badly 
entered numbers completed at this stage, since numbers need not 
be present on the Digitek cards. 

Alternatively, these sheets may simply be allowed to continue 
after the coded sheets; and these cards after the coded cards 
(i.e., no turning over of sheets, or rubber banding of cards). 
For small sections, it is not necessary to separate the uncoded 
from the coded responses, but for large sections (e.g., above 
30 students) it is very useful. (Naturally, completely blank sheets 
should be removed and destroyed when selected out.) 
This "separation" causes extra work for, the Digitek operator. 
It is most UBeful when handling large enrollment lecture courses 
which also have small lab and/or recitation sections (e.g., 
Phys 121). For such courses , regrettably, different questionnaires 
for the same section, or different sections for the same T/A 
sometimes get mixed together. Then remedial detective work is 
needed, which is made much easier by this Digitek separation 
process (see Sections 3.6c and 3.7). 



3. Each packet must be kept very carefully separate, and its 
cards kept with that packet* This is easy for those in enve- 
lopes, more difficult for those in boxes. 

4. Operator must be informed which packets will lead to 1 digitek 
card (40 or few questions), and which to 2 cards (41-80 questions) • 

The user will want to know the foirmat of the processed Digitek cards. 
In columns 1-9 the Digitek card carries the code number. It also carries 
a J or K in the 12th column depending on whether it is the first (or only) 
card corresponding to an individual answer sheet (up to the fixst 40 ques- . 
tions), or the second card corresponding to the 41-80th questions. Finally, 
in columns 13-52 it shows answers to individual questions. The user shotxld 
be warned that the order of the cards is the inverse of that of the answer 
sheets, after processing; i.e., the top answer sheet corresponds to the last 
pair of cards. Also note that a response "A" appears as a punched "0", 
"B" ^as "r\..,"E" as *'4", and no response is left blank. 

\^en the Digitek cards and original answer sheets are returned from the 
Computer Science Center, the answer sheets are saved since they possibly 
have written comments, but all further processing makes use only of the Digi- 
tek cards. Note that in the Digitek process only the responses to the ques- 
tions and the code number get transferred to the Digitek cards, but none of 
the other written identifying information on che answer sheets. It is best 
not to disturb the. older of the answer sheets when they are returned, and 
the Digitek card deck is being made up. This is so that if errors are later 
found by the program DATAREAD, they can sometimes be corrected by reference 
to the answer sheets (cf section 3,7 below). 

3*6 Ordering of Sections, Preparation of Header Cards, and 
Data Deck Assembly 

3.6A Ordering of Sections 

At some point, whether when first preparing the outside information on 
the manila envelopes and making header cards in advance (see section 3.3 
above), or after the return of the Digitek processed envelopes, all section 
envelopes must be sorted and ordered in an appropriate way. The following 
guidelines are ones presently used by PATS. They may be modified as appro- 
priate, but indicate certain rules. 

1. Physics must be separated from Astronomy, and each is handled as 
a completely separate job. 

2. Faculty are separated from teaching assistants, and each is handled 
as a completely separate job. [This may be modified when special 
cases arise, e.g., a faculty taember teaching a recitation. One may 
even duplicate the header and section cards in such a case, and 
have the section appear twice.] 

3. Within one job, (e.g., physics faculty) all sections belonging to 

a given questionnaire are handled together — i.e., an initial sort 
of the packet/ by questionnaire type must be performed by hand before 
assetd>ling the data deck. 
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4. Within a given questionnaire type sections should be arranged in 
ascending order of course number. Within a course number they 
should be arranged by ascending section number for faculty; but 
for teaching assistants within a given lecture section number they 
should be-'arranged by alphabetical order of teaching assist ant name , 
and for a given teaching assistant by ascending recitation or lab 
section number. [Check carefully that all sections are included.] 

5. Within one job, questionnaire types are ordered sequentially from 
1 through 7. 



3.6B Header Cards 

Recall that the Digitek cards only carry the code number as section 
identification. The Header cards enable the computer programs to be told 
the instructor's name, the enrollment, and any special conunents. More com- 
plete details are given in the description of DATAREAD. Probably the most 
convenient time to prepare header cards is when the packets are returned 
from Digitek processing, and using these packets as a source of all the in- 
formation. Alternatively, they may be provisionally prepared when first 
preparing the envelopes, with the expectation that corrections will have 
to be made in a few cases. 



As described in DATAREAD Ch. 4 , the header card format is variable. 
The format presently used by PATS is shown in Table 3.3, p. 3.9. 



In this case, the returned packets must be checked off on a master list of 
all sections, which is prepared when the packets are first made; so as to be 
aware of sections for which packets have been lost> not issued, etc. 
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TABLE 3.3 HEADER CARD FORMAT 





2 10 


11 12 




37 38 


39 73 


74 75 


76 77 78 


79 80 




* 


9 digit 
code no. 


A A 


instructor's name 


A A 


"comments" 


A A 


enrol- 
ment 


A A 





Notes: 

* — this identifies the card as a '^header". 
A — stands for a "space", i.e., a "blank". 

code number — (cols. 2-10) - this will generally be the saine as used when 
issuing questionnaires, but need not be (useful flexibility 
if an error in coding has occurred) . See further discussion 
in section 3.4 above. 

instructor's name — (cols. 13-36): 24 characters. This should be typed with 
last name first. First names and/or initials are optional 
after the last name. It is important that the first letter 
of the uame appear in the first designated column (13) , and 
that if the same instructor occurs more than once, his name 
be entered the same way each time. 

"comments" — (cols. 39-73): 35 characters. Generally this will be left 
blank, but it is available to warn of or explain special 
situations, e.g. "HONORS," "team-taught with X," "seminar, 
so questionnaire not issued," "taught by X, not Y," etc. 
Information may come from the returned packets. 

i^ nrolltOBnt — (cols. 76-78). This must be right justified [e.g. 17 must 

be entered either as A17 or 017], and is a (positive) integer be- 
tween 1 and 999. It should exclude students who have dropped the 
course, Either officially or uiiof ficially (I.e. taking an F) since 
it is used to determine the proportion ot the class that does re- 
spond. Again information may come from the returned packet. 
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KEYPUNCHING OF HEADER CARDS 

Witt, prepared, corresponding to the above format 

with numeric shift as appropriate. 

Such a drum card is the following: 

l\i . , (automatic duplication) 

col. 2-10: blank (numeric field) 

col. 11, 12: — (skip) 

*^°J'* alphanumerical field) 

. ^ (alphanumeric field continues) 

col. 37, 38: — (skip) 

*^°!"* . alphanumerical field) 

col. 40-73: A (alphanumeric field continues) 

col. 74, 75: — (skip) 

col. 76, 77, 78: blank (numeric field) 

col. 79, 80: — • (gkip) 



2. Crucial Remark. Header cards should be prepared on cards having a 
n^-cut right hand comer. This is so that they can be found easily ^n fhe 
^a deck, since digitek cards have the right hand comer lopped off. Ideally 

c!:: be^tld^^'iir""' '^'^ ' ''''' °' ^° ''^^ thf •pri'n'r''' 

deck^^Iv.^?J^f ^^'^^ ^^""^^ prepared (and later inserted into the data 
deck) even if for any reason no questionnaire answer sheets have been issued 
or returned, so that no digitek cards exist. One should stiU^e up deader 
nn^^nL^f fT *PP'°P5i*te explain the situation in the "comment." The 
purpose is to keep track of "missing" sections. 

« J* A 4^ ^^^"8 punched, an * (asterisk) must be 

T±ft''\ l . • "'^ automatically go to -numerical 

IIJIt inf enrolment. Punch the rest of the 

header information from the envelope. 

a) punch the 9 digit code number (no need to use "numerical shift.") 
button. i'««nediately type the instructor's name, then hit the "skip" 

* "comment" if needed, using numerical shift where needed, 
anformation nay be given for a "comment" on the envelope), then hU "sk?p!"' 
If no "comment" is needed, hit skip immediitely. 

4 -.4*4 A 9 ^""fi "enrolment" (no need for numerical shift) right 
iHtiged^[e.g. 17 must be punched 017 or "space" 17]. The header card is 
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5. Duplicate this header card using the "DUP" button. 

6. Type the next header card, starting at the code nuinber (the "aster- 
isk" will be punched automatically), and then proceding as in 4. 

In this way two copies of each header card are prepared "effortlessly." 
If convenient the complete pile of header cards should be read on a printer 
and checked for errors, but this is not essential. 

3.6C Assembly of Da a Deck 

One header card is placed in tront of its corresponding deck of digitek 
cards, the duplicate header is placed in another pile, the next header card 
is inserted, followed by its digitek deck, etc. The order in which sections 
must be assenibled has been discussed above (Section 3.6A). One thus assem- 
bles a data deck and a separate deck of (duplicate) header cards. If a sec- 
tion has no digitek cards, its header should still be inserted in the data 
deck, followed immediately by the next header card. 

When assembling the Digitek cards behind each header, it is very use- 
ful to visually inspect the separated answer sheets which bore no code num- 
ber and verify by written information on them that they indeed belong to 
the section. [Recall instruction 2 to the Digitek operator. Section 3.5 
above.] If any do not belong, it should not be too difficult to find the 
corresponding Digitek cards from the separated card deck and remove them for 
placement elsewhere as appropriate [remove both J and K cards if the ques- 
tionnaire has more than 40 questions.] The totally unidentifiable answer 
sheets and Digitek cards are considered to belong to the envelope where 
found. 

All the Verified" and unidentifiable cards are placed at the head of 
the Digitek cards for that section, right after the header. The answer 
sheets are left in the packet, so that they can later be read for comments. 

At this point only one questionnaire at a time within one job should 
be assembled into one subdata deck. These "subdecks" must each be preceded 
by " control cards" describing the questionnaires in more detail. These are 
explained in DATAREAD , Chapter 4. 

Such control cards also enable "overall" identifying information such 
as "FAJ^L 1971" or "PHYSICS, FACULTY" to be inserted into the record of each 
section within one job. 

* 

A complete deck for one job will then consist of: 

overall control cards 

control cards for questionnaire 1 

header 

Digitek cards 
header 

( CONT. on next page ) 

*As discussed in Appendix H.5.1 (p. H.5.^, it is possible to assemble and 
process a partial data deck, and subsequently add on an additional data 
deck. This (untested) feature could be useful if some sections are delayed 
for any reason. 
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(cont*) 
Digltek cards 

• 

control cards for questionnaire 2 
header 



Digitek cards 

3>6D Use of Header Card Deck 

This deck should be assenbled as the con^lete data deck is assembled^ 
in the sane order. It is useful to insert a fev blank cards between each 
questionnaire set and the next» and also at any other appropriate breaks 
(e«g* to divide undergraduate from graduate^ faculty from teaching assistants » 
etc.). 

This header card deck should then be printed (on fvll 120 character 
paper) as many tises as needed. It provides a **table of contents** for 
the output of DATAREAD and INITPRX. (AG6F0RM» which provides the final 
copy of output both for sections and aggregates^ does produce its own 
paginated table of contents.) This same table of contents forms the skeleton 
upon which the heirarchical aggregation tree is built up. Three copies 
of this list are useful for this latter purpose - one as an untouched origi- 
nal, one as a working sheet » and one as a basis for the final copy. A sam- 
ple is provided In Appendix C.l (see pat:ticularly top of p. C. 3). 

3. 7 Submittal and Output from DATAREAD and INITPRT 

We do not describe here the full details of submittal of data decks 
to DATAREAD. This is left to the separate chapter on these programs. The 
control cards carry Information on header and code number format » question- 
naire format (specifically number of questions on each questionnaire) » and 
overall descriptions^ such as FALL 1971 and PHYSICS^ FACULTY. All such 
Information must be Incorporated into the control cards. It is iiiq>erative 
that the interpretation of these control cards in the output of DATAREAD 
be checked. Errors In these cards can lead to erroneous output. 

A sample partial output of DATAREAD is shown in Appendix D.2. A 
fuller description is provided in the DATAREAD chapter. For each 
section the program lists each student's answer sheei: data* together with 
the code number the student entered (if any). The program alerts to any 
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5. Duplicate this header card using the *'DUP" button. 

6. Type the next header cardt starting at the code number (the "aster- 
isk" will be punched automatically) » anJ then preceding as in A. 

In this way two copies of each header card are prepared "effortlessly." 
If convenient the complete pile of header cards should be read on a printer 
2nd checked for errors, but this is not essential. 

3,6C Assembly of Data Deck 

One header card is placed in front of its corresponding deck of digitek 
cards, the duplicate header is placed in another pile, the nex^ header card 
is inserted, followed by its digitek deck, etc. The order in which sections 
must be assembled has been discussed above (Section 3.6A). One thus assem- 
bles a data deck and a s eparate deck of (duplicate) header card^. If a aec^ 
tion has no digitek cards, its header should still be Inserted in the data 
deck, followed immediately by the next header card. 

When assembling the Digitek cards behind each header, it is very use- 
ful to visually inspect the separated answer sheets vhich bore no code num*- 
ber and verify by written information on them that they indeed belong to 
the section. [Recall instruction 2 to the Digitek operator. Section 3.5 
above.] If any do not belong, it should not be too difficult to find the 
corresponding Digitek cards from the separated card 6dck and remove them for 
placement elsewhere as appropriate [remove both J and K cards i£ the ques«* 
tionnaire has more than 40 questions.] The totally unidentifiable answer 
sheets and Digitek cards are considered to belong to the envelope where 
found. 

All the **verified" and unidentifiable cards are placed at the head of 
the Digitek cards for that section, right after the header. The answer 
sheets are left in the packet, so that they can later be read for comments. 

At this point only one questionnaire at a time within one job should 
be assembled into one siibdata deck. These "subdecks" must each be preceded 
by "control cards" describing the questionnaires ;ln more detail. These are 
explained in DATAREAD^ Chapter 4. 

Such control cards also enable "overall" identifying information such 
as "FALL 1971" or "PHYSICS, FACULTY" to be inserted into the record of each 
section within one job. 

A complete deck^for one job will then consist of: 

overall control cards 

control cards for questionnaire 1 

header 

Digitek cards 
header 

( CONT. on next page ) 

*A8 discussed in Appendix H.5.1 (p. H.S4)>it is possible to assemble and 
process a partial data deck, and subsequently add on an additional data 
deck. This (untested) feature could be useful if some sections are delayed 
for any reason. 
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(cont.) 

Digitek cards 

control cards for questionnaire 2 
header 



Digitek cards 

3.6D Use of Header Card Deck 

This deck should be assenibled as the complete data deck is assembled, 
in the same order. It is useful to insert a few blank cards between each 
questionnaire set and the next, and also at any other appropriate breaks 
(e.g. to divide undergraduate from graduate, faculty from teaching assistants, 
etc*) . 

This header card deck should then be printed (on full 120 character 
paper) as many times as needed. It provides a "table of contents" for 
the output of DATAREAD and INITPRT. (AGGFOKM, which provides the final 
copy of output both for sections and aggregates, does produce its own 
paginated table of contents.) This same table of contents forms the skeleton 
upon which the heirarchical aggregation tree is built up. Three copies 
of this list are useful for this latter purpose - one as an untouched origi- 
nal, one as a working sheet, and one as a basis for the final copy. A sam- 
ple is provided in Appendix C.l (see particularly top of p. C.3). 

3. 7 Submittal and Output from DATAREAD and INITPRT 

We do not describe here the full details of submittal of data decks 
to DATAREAD. This is left to the separate chapter on these programs. The 
control cards carry information on header and code number format, question- 
naire format (specifically number of questions on each questionnaire), and 
overall descriptions, such as FALL 1971 and PHYSICS, FACULTY. All such 
Information must be incorporated into the control cards. It is imperative 
that the interpretation of these control cards in the output of DATAREAD 
be checked* Errors in these cards can lead to erroneous output. 

A sample partial output of DATAREAD is shown in Appendix D.2. A 
fuller description is provided in the DATAREAD chapter. For each 
section the program lists each student's answer sheet data, together with 
the code nuniber the student entered (if any). The program alerts to any 
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discrepancy between the header code number and the entered code number by 
a signal "ID CHECK". [Recall, Sections 3.6B and 3.4, that the header code 
need not be the same as that originally entered on the packet — though 
usually it will be.] All such discrepancies should be considered as inci- 
eating possible misplaced an»v:er sheets and Digitek cards. It is recommended 
that wherever possible one go back to the answer sheets in the packets and 
analyze all such discrepancies [excluding the obvious errors consisting of 
student social security numbers!] 

Other clues are available within the DATAREAD output, relating to the 
number of questions on the questionnaire versus the number on the Digitek 
cards. [The Digitek reader will read all answers up to a possible question 
40 on card J, and all answers between questic-^':, 41 and a possible 80 on 
card K. Of course card K is only punched ij[ the Digitek operator has been 
told to expect more than 40 answers. At present only questionnaire 5 pro- 
duces only card J, with, in principle, only 20 responses. DATAREAD only reads 
the number of responses it expects to find, and alerts if the expected JJJJ... 
or JKJKJK. • • sequence is broken. 

Suppose by error some "misplaced" responses to questionnaire 5 (lab 
section teaching assistant, 20 questions, only a J card) have been mixed 
with "correct" responses for the same course, same teaching assistant, same 
section but for questionnaire 4 (redtsection teaching assistant, 42 ques- 
tions, J and K cards). Since for example in Physics 121/122 the same 
teaching assistant does handle lab and recitation, such a mixup can and has 
occurred. Then DATAREAD will be expecting JKJK. . . and will find perhaps 
JKJJKJJK. ... It will alert to this error. Similarly it will alert if the 
opposite mixup has occurred. 

Another type of error that has occurred is the mixing up of responses 
for different sections taught by the same teaching assistant. The ID CHECK 
alert will give evidence of this. 

DATAREAD also enables the header card to be checked, furthermore it 
anticipates the possibility that by error no header card has been inserted 
between two Digitek sections. 

If for whatever reason cards have been found to be misplaced, the 
relevant Digitek cards must be physically located in the data deck and moved 
as appropriate. [Digitek cards are punched but not interpreted. However, since 
the first 9 columns represent the code number, these 9 coliimns should be 
identical for all cards in one section except for erroneous ones. Similarly, 
the J/K code in the 12th column corresponds to the "number" 1 or the "number" 
2 being punched (as well as an upper line that can be ignored). Hence it 
is not too difficult to find erroneously placed cards.] 

After one has verified and corrected the datadeck, one then is ready to 
prepare an output tape from DATAREAD, and to obtain actual accumulated 
questionnaire responr^e printed output, section by section, by use of INITPRT. 



It is similar to those given in Appendix Recall this is only an in- 

terim output, so that the format has not been edited for complete ease of 
reading* (Alternatively one can obtain section printout using AGGFORM, cf Gi 4) 

Notice that the program determines the number of students who abstain 
but do fill in at least the code number. This number is estimated , not 
from the re<;ponses to question 1, but from the minimum number , of "no 
response H indicated for any of the questions. Generally it will agree 
with the number of students leaving question 1 unmarked, i.e. ' "response f" 
but we have one documented case of a student who responded to other ques- 
tions after his lack of response to question 1 indicated that he was abstain- 
ing! [cf discussion in Section 4.4, p. 4.12] 

All distributions refer to students who responded to the full question- 
naire (though may have "not responded" to many individual questions). Thus 
for any question the number of indicated "no response = f" is a true indica- 
tion of the abstainers to that question. [Except for question 1, where one 
should also include the totally abstaining students.] 

The program prints a warning if fewer than 70Z of the "enrolled" stu- 
dents respond to the full questionnaire. This figure is adjustable. It 
plays a significant role in AGGFORM, and for this reason it is important 
that enrolment figures be up to date and exclude those "taking an F." 

One copy of the output from INITPRT is divided into sections and dis- 
tributed, together with a copy of the relevant questionnaire, to each eval- 
uated person, together with a request t^at he inform the PATS administrator 
within one week of any apparent errors. After such errors have been cor- 
rected, if necessary by rerunning DATAREAD and INITPRT and producing a cor- 
rected output tape, this output tape (or alternatively punched cards cor- 
responding to this tape) become? the input for AGGFORM. 

3. 8 Submittal and Output from AGGFORM 

3. 8 A Aggregates 

Once again we leave the full details of AGGFORM to a separate chapter. 
That chapter describes in detail what aggregates are, how they are specified, 
etc. Here we indicate guidelines that are followed, and present an actual 
example of a specification of a heirarchy of aggregates. We first list 
two of the special types of aggregate that arise. 

a) team teaching combinations . Suppose two instructors X and Y have 
team taught a single section A, and this section is later to be aggregated 
with other sections B, C. Then one presumably has obtained separate "sec- 
tion" responses A(X) and A(Y) in which the same students specifically eval- 
uated either X or Y. It is possible to compute an equivalent combined aggre- 
gate A(X+Y) in which the "average" response to X+Y appears. Symbolically 

A(X+Y) » J A(X) + ~ A(Y) . This combination section A(X+Y) is then on a 



Sample memo is given in Appendix D.3, p. Dg, 
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more equal footing with etc. » and is treated as an individual section 

in subsequent aggregations. 

b) complete descriptions (usually for teaching assistants). If a teach- 
ing assistant teaches more than one section in a course, one always prepares 
a complete description aggregate for that teaching assistant, being the 
aggregate of all his sections. 

All "combination" and "coissplete description" aggregates should always 
be made. Which higher aggregates are made is a matter of choice. The exam- 
ple given should indicate the type of choices involved. 

In general , all other aggregates will combine at least three individual 
sections. [To aggregate two sections U and V does not make it any "easier" 
to compare U with V, and the aggregate (U+V) will seem to have more "norma- 
tive" value. than is truly appropriate.] 

Initial aggregates should be chosen to provide meaningful combinations 
of "equivalent" or "peer group" sections. E.g., "all sections of course X," 
or "all first year physics major courses," etc. One is then likely to go 
on to "higher level" aggregates, which contain within them prevloiis aggre- 
gates, e.g. "all first and second year physics major courses;" and then even 
"all undergraduate physics major courses," and perhaps even higher. 

3, 8B Weightings, and Acceptance Criteria 

In the chapter on AGGFORM, .the different methods of "weighting" the 
various sections within an aggregate are described. Basically the only 
ones used by PATS are: 

a) P distribution - all included students weighted equally 

b) Q distribution - all included sections weighted equally 

c) R distribution - all included "components" weighted equally, 
[usually a "component" corresponds to a complete description, typically a 
teaching assistant]. 

When an aggregate is defined to include a section C, that section is in 
fact only included in the computation, if the participation rate for that 
section meets a (variable) level.* If the participation is below that level, 
no input from X is actually included in the aggregate. *[An "override" fea- 
ture is available to force the acceptance of designated sections.] 

Even when certain sections in a defined aggregate do themselves meet 
the participation level test, the aggregate is only printed in the output 
provided the number of accepted components is above a (variable) fraction 
of the total number of constituents defined to be in the aggregate. Typical- 
ly " components " are either sections or complete descriptions. Thus this 
criterion applies when an R distribution is to computed over teaching assis- 
tants, with respect to the fraction of teaching assistants with acceptable 
complete descriptions. [Again an "override" is available.] 

The purpose of these criteria (which are more ftilly described in the 
chapter on AGGPORUt is to ensure that only laibiased data gets propagated 
into aggregates. In submitting a list of desired aggregates, it is not 

*In fact, even this does not guarantee inclusion, if the section is itself 
part of a component of the aggregate. For then that component must pass a 
separate participation criterion. 

tSee Section 5.2g(p.5.3). Part I Section 2. 5, (p. 2.5) gives the fullest dis- 
cussion of these points. 
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necessary to consider such questions in advance, unless one is aware of 
the need to use the override for some special cases. Generally on the first 
submittal one will not put in such overrides. One can always go back and 
resubmit with appropriate overrides. 

3.8C Specification of Aggregates 

In order to prepare all necessary instructions for AGGFORM, as described 
in that Chapter, one m\ist know which aggregates are to be computed, each 
such aggregate must be given a number, and each such aggregate must be 
given a name or title which describes it adequately. It is also possible 
to add "comments" to each aggregate, which are not part of the title, but 
explain specific features. 

Aggregates are printed out in the order in which they are numbered. 
The numbering scheme need not use every integer in sequence. Hence, to 
handle possible later insertions or errors, it is a good practice to number 
aggregates by tens within a particular group, with larger gaps between 
groups . 

The easiest way to indicate the content of each aggregate is to draw 
a heirarchical tree, using the header card deck printout (section 3.6 D) . 
In Appendix' C.l we show such a sample heirarchical tree, and in Appendix 
C.2 we give the corresponding list of names and comments. A careful 
study of these appendices should make the task self-evident. [The meaning 
of the numbers. In parentheses is explained in section 3. 8D.] 

Notice of course the following restrictions: 

a) aggregates can only be made within one "job", 

b) within one job they can only be made within one questionnaire type. 

It is therefore possible to use the same numbers for aggregates within 
separate jobs [but this may lead to confusion]. Within one job all aggre- 
gates must have different numbers. A possible scheme is to use 1000 - 1999 
for questionnaire 1 aggregates, 2000 - 2999 for questionnaire 2, etc. 

Let us now look at the heirarchical tree and headers in detail Appen- 
dix C.l (also see Appendix 0.2) . On the first pages of the tree, there 
is little that is not self-evident. Since each "section" represents a sepa- 
rate faculty lecture group, only P and Q weightings are used. In general, 
aggregates contain at least 3 sections, except for 3110, which corresponds 
to a separate subsequence of courses. 

On the second page of the tree, things become more coii?>licated. Notice 
first the use of comments when appropriate. Next, consider the 6000 se- 
quence of aggregates. Course 429 is actually the same course as 621, so 
that 429010162 is really the same section as 621010162, but contains responses 
from different students. Aggregate 6010 puts these responses together, and 
leads to a "complete description" of Martin. Unfortunately, it is counted 
as two sections [AGGFORM as yet has no facility to add parts of one section 

■r 

We urge the reader to study Part I Sections 2.4 & 2.5 (p2.2-2.6) to obtain a 
full understanding of the effects of the various cut-off criteria. A "tech- 
nical" point, and a "technical" solution, here had profound practical reper- 
cussions which should be fully understood. 
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and produce one section, but such a modification is fairly easy to program.] 
Actually Martin, Zom and Pechacek taught separate groups of students, and 
each should be counted equally, and each as one section* Aggregate 6030 
assembles these three sections into one* But since aggregate 6010 and 6020 are 
each counted erroneously as 2 sections, Q weighting by "sections would be 
incorrect. Hence one \i3es the complete P/Q/R weighting. The R weighting 
treats aggregate 6010, aggregate 6020, and section 62101061 Zom equally. A 
••comment" is included in the "description" of Aggregate 6030 tp warn about 
the incorrectness of the Q weighting. 

Let us then turn to the 7000 series. In this case, as shown on the 
"comments," sections are team taught. Hence one first makes . individual 
combination sections. Aggregates 7000 - 7030 and 7100 - 7120. Each of these 
count as one section. From these, one then makes course aggregates 7060 
and 7160, in the regular way. Finally an aggregate over all such courses 
is performed to obtain 7200. But one also wishes to obtain complete descrip- 
tions of each professor. This is done on a separate tree with aggregates 
7040, 7050, 7140 and 7150. 

The above examples should be sufficient to indicate the types of flex- 
ibility available. Each aggregate is given a title, and one may add to 
this title "comment cards" with full explanations of any irregularities, etc. 
Some such comments are shown in Appendix C.2. 

3.8D AGGFORM Output and Distribution 

AGGFORM can produce four separate outputs for a given job 

a) All individual sections 

b) All aggregates 

c) Multiple copies of aggregates (MULTIPLEMANIA) 

d) All complete descriptions. 

Each is provided with a table of contents, and each may be printed out as 

often aa is desired. [Samples of (a) and (b) are given in Appendices E.l and E.2.} 

Actually there are many more possible selections such as ''all aggregates 
within an aggregate number interval." These are explained in the AGGFORM 
chapter. 

Presumably (a) and (b) are self explanatory. Output (d) is not par- 
ticularly used by PATS, and hence is explained in .the chapter on AGGFORM. 
Multiplemania contains multiple copies of aggregates, according to any 
desired scheme, which however must be specified when the aggregates are 
first defined and formed. The usual scheme is determined from the heirar- 
chical tree, in such a way that one copy of Multiplemania provides the con- 
tents of a "conq>lete heirardiy ." TKis concept is most readily understood 
if one refers to the heirarchical tree in Appendix C.l. 

Aggregate 2000 is of immediate interest to three people, who however 
are also interested in Aggregate 2040. Aggregate 2040 is of interest to 
13 people. A complete heirarchy produces enough (varying) copies of each 
aggregate so that each person can receive one copy of each of the aggregates 
in which he appears. [Notice that "higher" aggregates such as 3250 need to 
be produced in many more copies than do lower level aggregates, such as 3000.] 
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It is clear that the heirarchical tree contains within itself the information 
needed to determine the number of copies needed. Nonetheless, in setting 
up the control cards for AGGFORM, one must specify this number explicitly. 
The easiest way is as we have done on the List of Aggregate Titles .Appendix 
C.2. The number in parentheses is the number of total copies of each 

aggregate that is required. Generally (but contrast 6010) this equals the 
number of original sections involved, and is cumulative. Thus since (Aggre- 
gate 3260)" = "(Aggregate 3120) + (Aggregate 3250), a similar equation applie.) 
for the number of copies needed. 

Having put such effort into computing all the output of PATS, it is es- 
sential to squeeze the fruit fully and distribute the output for maximum 
accessibility, ease of comparison, and (hopefully) impact. Within one job. 
each of all sections" and "all aggregates" can easily run to 200 - 300 
pages, and hence they should be put in suitable covers. 

The full distribution used by PATS is given in Appendix G.l. 
In each case, a covering memo (Appendix G.2), and a copy of the appro- 
priate questionnaire or questionnaires is also distributed (do not expect 
people to keep these from the INIT-PRT distribution). 

The distribution under "A" is self-explanatory. Within one job, one 
copy of (a) all sections, plus one copy of (b) all aggregates, produces 
one copy of A. 

To make one distribution under B (e.g. to the individual) one copy 
of a, and one copy of "c" is needed. Each of "a" and "c" must be hand 
separated and sorted - BUT THE EFFORT IS WORTHWHILE. The tables of contents 
of (a) and (c) together with the first pages of each aggregate, give assis- 
tance in performing the sorting. Each individual receives copies of all 
data in which he features as a component. 

. The distribution under C may be explained by referring to the table of 
aggregates, Appendix C. There is a course subcommittee whose juris- 

diction is all courses within aggregate 3070, and another subcommittee whose 
jurisdiction is all courses within aggregate 3080. There is also a committee 
whose jurisdiction is defined by aggregate 3100, which includes both 3070 
and 3080. Each committee or subcommittee receives one copy of each section 
or aggregate within its jurisdiction, plus one copy of each aggregate in 
which any of the components comes within its jurisdiction. 

The distribution uoder D is similar to C, except that one replaces 
•committee or subcommittee" by "facia ty member." 

Within physics, we find that a complete distribution under C and D 
requires 

all sections, faculty 2 copies 

all sections, teaching assistants 2 copies 

all aggregates, faculty 1 copy 

all aggregates, teaching assistants 1 copy 

multiplemania, faculty 1 copy 

multiplemania, teaching assistants l copy 
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which leaves a lot of surplxis (scrap) output. [This may not always work 
out satlsfactorllyt In which case one may assign different numbers of 
copies in MULTIPLEMANIA. This should be thought through in advance of 
execution of AGGFORM.] 

It should be remarked that the major part of ::he actual CPU computing 
time is spent on assembling the various pri.nt files; more than: in actually 
computing the aggregates themselves. To assemble a large Mult^lplemania 
may take 10 - 15 minutes » and require 500 or 750 tracks (i.e. It will over* 
flow on a regular size FASTRAND file). Subsequent ^'printed**, copies of 
Multiplemania use no CPU time» since they are ^'SYMMED** out» but tie up the 
CSC Printers. (Multiplemania can ''easily*^ exceed 1000 pages and lead to 
an 8 inch stack of paper.) Thu8» thou^t should be given to the necessity 
for Multiplemania » and to the nunber of copies of aggregates it should 
contain. Possible economy measures include: 

a) Let Multiplemania contain one less than the number of copies of 
each aggregate needed » and supplement it by use of one copy of **all aggre** 
gates.** This ^'solution** makes the final collation and distribution slightly 
more complicated. 

b) Arrange that each instructor receive copies of only the two or three 
^'lowest** aggregates In which he appears. (These will be the ones closest 

to representing his peer group.) Refer instructor to the public display 
of **all aggregates** for higher aggregates. This **solution** is the most sat** 
isfactory and efficatiou8» since very *1iigh** aggregates (e.g. all under- 
graduate major and graduate courses) are somewhat meaningless » yet are the 
ones that consume the greatest CPU and printing time in assembling Multiple- 
mania» since they contain so laany instructors and hence **need** to appear 
so many times. 

c) Only assenble and .print one copy each of **all sections** and **all 
aggregates.** Obtain all other needs by use of a reducing Xerox. Alterna- 
tively use Xerox to prepare all needs for distribution to individuals 
(Multiplemania) but print as many copies as needed for permanent records of 
**all sections** and **all aggregates.** 

3.9 Suggested Improvements 

As stated before^ this report describes the present form of ?ATS. In 
the light of the two large implementations of Spring and Fall 1971^ various 
possible improvements have suggested themselves. These are described in 
Appendix J. In some cases their full significance will only become 
clear after a reading of the chapter on A6GF0RM. 

3.10 Optimal Timet Ale of Operations 

The description of PATS which has been given here followed a logical 
temporal sequence. However » in order to expedite final output » it is pos- 
sible to prepare various stages in advance. In particular^ we believe that 
every effort should be made to distribute the aggregate results at the same 
time as the individual section results. For this reason^ we strongly recom- 
mend that the following stages in PATS be performed prior to the actual issue 
of the questionnaires to the students: 
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1. Preparation of one set of Header Cards, but with enrolment 

left blank, (cf. Section 3.6B) 
1. Preparation of control cards for DATAREAD and INITPKT.(cf. Chnp. 
3* Preparation of aggregation tree and titles (cf Section 3.8C) 
4. Preparation of all control cards for all executions of 

AGGFORM. (cf. Chap. 5) 

The reader may object that since the enrolment figures (which are cer- 
tain to change) must still be added to the header cards, that some other 
facts about some sections are likely to change, that some comments may need 
to be added> and that some other unforeseen complications may arise, all 
the above effort is wasted. But this is not true, since such changes can 
be anticipated and hence allowed for. 

Actually, one can use the header card deck (with no digitek cards) as 
a "trial" data deck, and use DATAREAD and INITPRT to produce a trial con- 
densed data tape. This tape can then be used as input for AGGFORM, and 
hence one can test the control card decks for AGGFORM before having any 
questionnaire data. In this way one can check in advance whether each aggre- 
gate is indeed made up of the correct constituents. As previously stated, 
one is nearly certain to find errors, which can hence be corrected in ad- 
vance. This debugging can save much time later when actually executing 
AGGFORM. Furthermore preparation of all the control cards in advance will 
save much time when actual data exists. 

When the actual data is received (i.e. packets, and then the digitek 
cards), one can then complete and update the header cards, by duplicating 
them at a keypunch while inserting alterations and enrolments as appropriate, 
or alternatively one can simply prepare two new header card decks. One 
can also modify the AGGFORM control cards as necessary at this time; but only 
a few control cards are likely to need any changes. (We remark that a 
skilled programmer can actually put the AGGFORM control cards on file in the 
1108, and update them by editing via TELETYPE.) 

Thus fay performing as many stages as possible in advance, one can greatly 
expedite the time at which final AGGFORM output Is available. [Note that 
this idea has not actually been put to the test!] 



3*11 Conclusion 

This completes the. overall detailed description of PATS. The remaining 
chapters are rather more technical, and deal with submittal of actual runs 
under DATAREAD, INITPRT and AGGFORM. They are intended for the person or 
persons actually submitting runs to the computer, and presume some general 
computing experience. 



CHAPTER 4, INITIAL COMPUTER PROCESSING - DATAREAD AND INITPBI 



4> I General Introduction & Outline of Chapter^ 

This chapter consists of four sections. The first provides a general 
overview of DATAREAD and INITPRT. Section 4.2 discusses some of the functions 
performed by DATAREAD. Section 4,3 (together with related appendices) Is 
a detailed operating maxmual for the program * DATAREAD. It need only be 
read by the person actually handling the computer runs of DATAREAD and 
INITPRT. It is highly "technical*^ and need not concern the gener^ PATS 
administrator. Finally section 4.4 discusses a minor difficulty concerning 
students who abstain. 

Let us ^expand somewhat on these various topics. We discuss the general 
nature of the functions performed by the program DATAREAD , since such functions 
must form an integral part of any such computerized survey procedure . Then 
in section 4.3 we describe the actual program DATAREAD in more detail, in 
particular disctxssing the control cards and instructions for executing DATAREAD. 
This description (and the program itself) have deliberately been written in 
a very general way, more general than is needed for the present version of 
PATS. This generality makes the description somewhat harder to follow, but 
enables the program to handle rather considerable changes in the PATS pro- 
cedure. In fact the write up is deliberately written as a completely self* 
contained dociment, and does not refer specifically to PATS. The write up 
can be more readily understood if one makes specific reference to the actual 
PATS-DATAREAD exa]iq[>les given in Appendix D. 1 (which refers to questionnaire 2). 

Versions of DATAREAD exist for both the IBM 360 and the UNIVAC 1108. If 
one tises the UNIVAC 1108 computer, there is then no need to use the program 
INITFRT. This is because the output tape of DATAREAD can be used directly 
as input for A6GF0RM, which itself will produce printed output for all 
individual input sections (if necessary, one can run AGGFORMl without creation 
of any aggregates in order to tbtaln this rapidly). Since PATS is presently 
run at ^1 stages on the UNIVAC 1108, it follows that the program INITPRT 
is superfluous. Eence this report does not address itself further to operating 
details for INITPRT.* 



Let us return to our main concern, the program DATAREAD. Appendix D«l 
presents a sample listing of control cards a) directly as input in the run deck, 
b) as "directly" printed out with only minor format changes, and c) as inter- 
preted by the program. In any actual nm, it is essential that the "inter- 
pretation" be checked against the actual questionnaire format, for each ques- 
tionnaire. For an error in such a control card can be made quite easily, and 
would lead to erroneous output for all sectional Such a check should be made 
by the overall PATS administrator, and is really quite straightforward. (As 
can be seen from Appendix D.l.c, the "interpretations" as printed out are 
reasonably self-explanatory, and hence can be checked without any knowledge 
of the control card structure.) 

Appendix D.2»a (page D4) shows a part of a sample section DATAREAD 
printout. It connences with descriptive information taken from the header for 
that section, then lists the "contents" of each student answer sheet. It 



•when PATS tooA ^lut -implemented, both PATAREAP and IWITPRT toe/te >tan on 
an IBM 360. It Is ioK that hlstofUcal KOMon that /teXe/tencei ha\)t been wade to 
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"reads" the contents by using the interpretation information already referred 
to above, to select certain colinnns from those on the Digitek card deck. Each 
student is referred to as a "data set". For each student there is the IP 
number he entered, followed by a listing of the entries in "each" coluiin of 
the answer sheet (but again as interpreted from selected columns of the Digitek 
cards). A blank means no entry was made and 0,1, 2, 3, A correspond to (a) (b) 
(c) (d) (e) respectively. 

In the right hand margin of this DATAR&VD printout » one sees "ID check" 
at each data set, since the student*entered ID disagrees with the header ID. 
Similarly one can get other warnings referring to the sequence of Digitek 
cards (cf also section 3.7, p. 3.12). Clearly all such warnings must be 
followed up and dealt with as discussed in section 3.7 (p. 3. 12) before mak- 
ing the final DATAREAD run. There is no need to "correct" misentered ID 
numbers if one is "reasonably" sure that the Digitek cards are indeed in the 
correct sections; on the other hand» all "sequence" errors must be corrected 
completely before the final run. Any such errors can always be corrected by 
comparison with the original answer sheets. 

In fact, it is possible to "cheat" a little here. As long as the correct 
sequence Is followed, e,g. JKJKJK..., and each member of a JK pair has the 
8^ ID number, there is really no need to ensure that the J and K cards be- 
long to the same answer sheet, provided both have entries. This is because 
all later programs only look at "summed" responses. [On the other hand, if it 
is anticipated that "correlation" studies of student responses will be made 
using the Digitek decks at some future time, this shortcut is not allowed.] 

At the same time that DATAREAD reads the input Digitek deck, it tabulates 
the responses for each section, question by question, by summing the number of 
students who made each response (a) (b) (c) (d) , (e) or "blank". These summed 
distributions of responses are put onto the output file of DATAREAD, (usually 
on magnetic tape). Appendix D.2.b (p.D.5) shows the format of one such sec- 
tion of the output file. (The whole file is in regular FORTRAN file format, 
with no "marks" or other divisions between sections.) 

The output file of DATAREAD becomes the input for INITPRT, and also 
(independently) for AGCFORM. INITPRT printout looks similar to individual 
section AGGFORM printout (cf Appendix E.l) and hence we do not show a sample. 

As discussed at the end of section 3.7 (p. 3.1A), one copy of the print- 
out of the individual sections is distributed to individual instructors, to- 
gether with a cover memo, .a sample of which is given in Appendix D.3, p. D^". 
One should either distribute conqplete section information together with a 
questionnaire (and request that these be kept for future reference) or simply 
distribute the first "descriptive" page of the section printout. The latter 
procedure has the advantage that final section data, i.e. the actual response 
distributions, can be issued simultaneously with the relevant aggregate dis- 
tributions, which provides maximum "normative" and psychological impact — the 
disadvantage is that if an error has crept into the Digitek processing or 
elsewhere, it may not be spotted until the full section data are distributed. 
[Naturally the cautious PATS administrator will do some spot checks "by hand" 
using the original answer sheets, preferably for each questionnaire!] 



This coiq)lete8 the overview of the main parts of Chapter A and related 
appendices* In section 4.4 we briefly discuss a ndnor technical point about 
how PATS presently (somewhat unsatisfactorily) estimates the number of stu- 
dents who abstain from responding to the questionnaire* 

The next topic to be discussed is the program AGGFORM^ which forms the 
content of Chapter 5* 

r 

4*2 Some Operative Functions Performed by the Program DATA1U5AD* 

PAtS uses header cards as described in Chapter 3 to divide up the 
digitek data cards into the basic sections* These headers carry the sec- 
tion ID code nunbersy as well as detailed information concerning each sec- 
tion (e*g* instructor's name» enrolment» etc*) (see section 3*6» pages 
3^ 7*3^ 12) » Thus, each header card serves three independent purposes: 

a) It Indicates a division of the digitek cards into a single 
section* We refer to this as the dividing function* 

b) It provides an ID code nunber for that section* 

c) It carries as^lifylng identification information which inter- 
prets the ID code nunber* We refer to this as ^'directory** information* 

These three functions could well be performed in different ways* In 
particular^ the directory information could instead be **inputted** in a 
separate part of the data handling* by actually preparing and reading in 
to the computer such a directory* (With coiq)uterized preregist ration* the 
registrar should be able to assist in providing punched directory cards 
automatically*) 

The dividing function could actually be handled in one of several ways: 

i) as at present in PATS* The header cards define sections* and 
carry the ID nunbers to be associated with the sections (with the option of 
also carrying directory information* if desired)* 

ii) by using divider cards which are all identical* but carry no 
ID nunber or directory information (e*g* siiiq)ly an **asterisk*' in column 1)* or 

iii) by using no divider cards at all* but using the computer to sort 
the digitek cards into sections* 

In cases (ii) and (iii) the ID nunbers on the digitek cards would be used 
by the computer to determine the appropriate sections and ID nunbers* 

The reader may well object that one knows that students make errors 
in entering the ID nunbers* However* one can program to allow for^this* 
The DATAREAD program has the flexibility to operate under options (ii) and 
(iii) above* The programming assumes that student errors will not occur 
systematically* Thue under option (ill)* DATAREAD sorts and looks for pat- 
terns* and is able to grot9 data cards into sections* Specifically* let 



individual letters a, b, c... represent the ID numbers as entered by stu- 
dents (with A representing a ••blank*' number). A sequence 

abaadiaal dddtbddAd| fgggg. • • 

would be divided by the DATAREAD program as indicated by the vertical lines. 
It is probable that ••£•• was a misentered ••d^^ or ••g/^ The computer output 
provides the operator sufficient information so that he can then <lecide 
how f should be handled. The program run in this mode will surely be run 
in several passes. (Even with header cards present, one does move digitek 
cards to different sections, and make other corrections, so that more than 
one pass is made.) 

As presently written DATAREAD can be run either as in PATS option (i), 
under scheme (iii), or even under a "wlx^^ of options (i) and (iii). fit 
could also be modified to operate under scheme (li).] This flexibility is 
what ••requires^' the program description to be rather general. (However, 
reference to appendices Dl and D2 should clarify the description.) In the 
••general description^^ In the following section, one sees (In the second 
paragraph) that the program sorts (or divides) by the ID nuisber on the 
digitek data cards automatically, with divider (or header) cards being op- 
tional. Hence also the specific contents of the header cards are optional. 
The program as presently written can be na so that sorting is done purely 
on the basis of the headers; or (with no headers In the deck) purely on 
the basis of the digitek card ID's; or primarily by the program on the basis 
of the digitek card ID's, with header cards overriding and forcing new sec- 
tions when such headers are found. The sample decks have the torm appropriate 
for sorting purely by headers. 

Besides these specialized ftmctlons, DATAREAD automatically performs the 
basic functions of: 

a) determining the number of student answer sheets for each section 
(thus these do not need to be hand counted) , 

b) determining the summed distributions of answers to each question, 

c) printing a ••listlng^^ of each student *s responses, together with 
some automatic checks, for validation purposes, 

d) preparing an output file which contains all the stmmed distribu- 
tions. 
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I A, 3 The Technical Degcrlotlon of the Protr— DATAREAD A.5 

University of Maryland Cyclotron Laboratory 
I OIGITEK CARD PROCESSING PROGRAM 

I GENERAl'oESCRIPmJ fThU iJiCti^n ihouU Jl^ 

................... I PATS nm dzck gAven <n Appendix V.U 

I This program was developed as a means to tabulate responses to questionnaires 
processed by a Oinitek optical paxe reader which produces punched card output. The 
program considers each set of responses as a unit of data '<data set) which ir 

f verified for consistency of content and format. Data sets hkvins like content are 
grouped together, excluding rejects, into sections, . and then the individual 
responses are summed within each section to form an output fi le of reduced 
dimensions. 

I Although sorting by content is performed automatically, dividers (header cards) 

may be manually inserted within the input file to force section groupings and 
provide descrptive information. Sorting may be performed entirely by the header 

p cards, entirely by the program, or primarily by the progra:n with headers overriding. 

A l«r«e measure of flexibility Is provided by run-tine specification of the 
header and data card formats and header content; the program is insensitive to such 
T- things as number of forms being processed, programming of Olgitek machine, number of 

j responses in a questionnaire, etc. information of this type which is provided by 

the user Is, whenever pertinent, passed on to the output file. Thus the latter 
becomes self-descriptive, i.e., subsequent users of the output file who are aware of 
r. Its fonnat (which is invariant) may access information without re-specifying any 



r 
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INPUT FI LE {TkU ducfUbti tkt eomptUt bvpvJt iatx dtek putsfioMly dUcuutd in StcUon 5.<C,p3. If)* 

I I . input to thft prosrara may ba logically separated Into two parts: 

I (A) data cards (as produced by Dlgltek) and headers (optiona?)^ Inserted manually 

jr. Into the data deck; and 

I } CB) program descriptor and command cards^ which describe the input and perform 



simple control functions* 



(Al) data cards: Itht VigUdt coAiU - c<. ^tUion 3.S/pwUcUaHJUf p. 5.7.»* 
The content of the data cards is flxed^ as follows: 

le Identification field- Ipmintiif tht iiAkt 9 eotwm) 

0-12 characters (the limit is arbitrary and can be easily changed in the 
program) located contiguously anywhere on the card 

The 10 field is useful in providing coded information that Is unique to 
•ach section but common to the data sets within a section; it provides 
the basis for section groupings in the absence of header cacds. 

f2e Questionnaire i d€ntt fitr ^ IpHUtntty tht Uk ectum) 
1-6 characters located contiguously anywhere on the card to identify the 
questionnaire; may be part of the ID 

IS. Sequence field - Ipftutntty <l3 oKKJintkt Htk eatwrn] 
a single character^ located anywhere on the card 

%• Data field - {ptMAvM^ ktofitk at tkt 13th eotum} 

f represent six possible responses to Items on the questionnai- e, and can 

be easily changed in the program. 



ERIC | * ^ oitiiu in iXalAA typt KtitA ^pteliicaUy to PATS, omf do not p^tt oi tkt £o ^^^^ fMgnam 
"^f^^ ducMiptiotu 
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A data set may contain 1-5 cards^ each possessing all of thn o')Ovo 
fields* The lengthy content^ and position of thp ID field should be constant 
within a data set. Normally each card v/lll have a unique soquencp chorncior, 
which may appear in any position* The data fieli on encb cnrd nay l>-vo .\ 
d I f ferent length and position^ but the total number of data characters is 
limited to 99. (This may be easily changed in the progra-n). 



(A2) header cards: [c^ izcX^on 3.6E and tabtz 3.3, p 3.9] 

the content of the 'leader cards Is variable^ and may contain the following: 

1. Recognition character - {pKo^zkMy an oj^toxibk *) 

a single character^ specified by a program command card (see below) which 
Is used to distinguish among header/ data^ and command cards; must be 
present In the first position on the card. 

1. Identification field- {pKt&zyMy co£umw4 2-10) 

optional; same as data card: The header ID will override subsequent data 
id's. 

5. Questionnaire Identifier- {pKt^zrvtty cjotmn 9) 
optional; same as data card 

k. Size field - Ithz znnjoJbnznt** - pKo^enttn cotmn6 76-7i) 

optional 3 digit number to indicate the number of possible respondents to 
the ques tlonnai re^ located anywhere on the card, but with the digits 
right Justified within the field 

5. Optional fields - [pK'^znJtZy thz itU^tniicXoK^ ^ name, and "commeittA") 

up to 5 additional fields containing general Information to be inserted 
Into the output flle^ and located anywhere on the card 

With the exception of the optional fields^ all header cards should be 
consistent regarding which Information is present, 

?B) gpmman^ and descriptor cards: {cj{ Appendix P.J ioK an aciwxZ PATS example) 

The primary, purpose of descriptor cards Is to describe the format of the data 
and header cards. Command cards are used to indicate the presence of 
specific descriptor cards (e.g. "HEADER" command)/ to signal the program to 
take some action (e.g. "START" command)/ or to set a parameter value within 
the program (e.g. "UNIT" command). The format of the command cards 
themselves Is fixed/ as follows: 





(1) 


recognition character 


(col.l) 




(2) 


6-character label 


(col. 2-7) 




(3) 


three 2-dlgft numbers 


(col. 8-9, 10-11, 12-13) 


i 




one blank 


(col.l«») 


i 
i 


(5) 


ZU-character identifier 


(col . 15-38) 



Since the meaning of each field varies according to command type, and the 
number of different types Is small/ each type is described separately below. 
Upper case Is used to indicate that a name Is to be punched on the card 
exactly as It appears in the example. The vertical lines are used as field 
del iml ters* 

■leader comma nd apd assop iated dpscrlptnr gard*^ 

The first card is: 
coKl 8 15 

|$|HEADER| I I II I 

The first character on the first card encountered by the program defines 
the recognition character for all subsequent command cards; a dollar sign Is 
used for Illustrative purposes. The header command contains' only a label 
field/ and indicates that all subsequent cards up to the next command card 
will describe header Information. (Tfie doiZax ^ign JU oied in thz acXiiaJi examp£eA) 



er|c 



The set of descriptor cards following the header command card is used t.j 
establish two specific header fields and a variable number of general header 
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fields. All fields hove associated with them a 2«»-character Identifier v;hich 
Is supplied In columns 15-38 of each card. The program rcccf,nl2es the 
Identifiers "SIZE" and "ID" as being special cases. (The order In which these desc- 
riptors are listed is imnaterlal.) 
Size Descriptor card ; This has the fom: 

arm — — 8 — 15 

1*1 1 ImmlnnI ISiZE I 

The "SIZE" descriptor card specifies a 3-dlglt field on the header cards 
from columns mm to nn as previously described. In addition^ a non-blank 
character In column 1 defines the recognition character for all header cards 
In the data; this character (an asterisk Is used here as an example) is also 
presenc on all subsequent descriptor cards which pertain to he'tider format. 

ID Descriptor cards : A set of theaewill have the fom; 

col.l 8 15 



1*1 
i I 
I I 
I I 
I I 
i I 



mmlnni | ID j 

mmlnnj I subl j 

minlnnj |sub2 j 

mmjnnj |sub3 | 

mmjnnl j sub<* | 

mmjnnl |sub5 j 



The "ID" descriptor card specifies the variable length field on the 
header cards from columns mn to nn as previously described. From 0-5 cards 
having a blank recognition character follow this card to establish subflelds 
within the ID^ designated by the names in the identifier field (subl^ ...^ 
sub5 In the above example). The specifications for these subflelds (mm to 
nn) refer to rel ati v^ positions v;i thi n the ID. 

General Field Cards: These are used to locate the optiaial fields; and can also be ixsed to 
l^ut^ special iijfbnnation into each record. (In PATS tiUs Is u6td ioK tho, datt and Aujbc£a&A) 

1*1 I ImmlnnI | name | 

The general header fields are specified by 0-5 descriptor cards following 
the last ID descriptor card (If present). Each card sets uo a variable 
'length field on the header card in columns mm to nn which is assigned the 
name given In the Indentlfier field. 



2. Data command and associated descriptor cards 
c^f .^^^ card ^s: - . 

|$|DATA I I I I I I 

The DATA command contains only a label field. Two types of descriptor 
cards follow the DATA command to specify the exact format of Information on 
the Dlgltek cards. Each type Is recognized by Its Identifier^ which must be 
•MD" or "SEQUENCE" as Illustrated: 

ID Caixi : This has the fom: 

coKl 8 15 

I I I ImmlnnI flD j 

This specifies that the ID Is found In columns mm to nn on the Dlgltek 
cards. (It Is assumed that any ID subflelds are sped fled wl th the ID 
descriptor card that follows the HEADER command card.) 

Sequence Cards : These vaiy ftxxn questionnaire to questionnaire, and have the fom: 

coKl 8 15 

Id Ml ImmlnnI ISEQUENCE | 

Each "SEQUENCE" card contains a sequence character^ c^ which the program 
win expect to find In column 11 on the corresponding Dlgltek car^l^ along 

!fJiJ!lc^^f ^^^^9 There must be as many sequence descriptor 

cards as there are Dlgltek data cards per data set. 
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3* .^P,'!!^ .gPr^'^^^l-^g^L.^ssocIatctj descriptor cnrds ^ This refers to the questlcrr.-il:^ 

{In PATS the quttUonndOics OAC "rtomccf' by tht ntmbcM 1 tlwoiish 7). The first card is: 
coKl 8 IS 

ISIFORM I i I I I name I 

Use of the FORM command will supply a 1^6 character nane for the 
questionnaire. It may be followed by descriptor cards to indicate the 
location of fields on header and data cards/ respectively^ as shown: 

col. 1 8 15 

|«| I |min|nn| Iname j 

I I 1 jminjnnl I name I 



ii. Unit command card 

This card has the fonn: 

col.l 8 15 

|$|UNIT InnI j I iname j 

The UNIT command is an optional device to respecify the assignment of I/O 
units used by the proi^ran. "nn" is a FORTRAN logical unit number, and "name" 
represents Its corresponding function. Def aul t valuos are preset to the 
following which are appropriate for the IBM 360. (For the Ihlvac 1108 one irust set PUNCH 
at the value 01, but the other default values are also approoriate for the 1108, ) 

name value function 



CARD 05 card input file 

PUNCH 07 ' output file* (01 for 1108!) 

PRINT 06 printed output 

SCRATCH 03 Intermediate storage unit 

5. Start co'iwand card 

IhiR card has the fonn: 

coKl 8 15 

ISISTART I I I t I I 

The function of the START conaand is to mark the end of the set of 
coiivaand cards, and signal the program to begin processing data cards. 



6 . End command card 

Ihis card has the f om: 

coul 8 15 

UIEND t t I I 1 I 

The END command marks the end of data card input for a given 
questionnaire, and signals the program to expect the next card to be a 
command card. The START and END conmands serve as delimiters for the data 
cards associated with a single questionnaire. When multiple questionnaires 
are processed within a single job, fields may be re-speclfled by com-nand 
cards located after the END comnand. 



7. Sto p c omnand card 
Th^s £ard has fcm: 

l$|3T0P t 1 I I i I 

When an END command Is follov^ed by a STOP command card, the program will 
terminate operation. 

''Punch >te5e.t4 to thz output iitt (c<.p4.9), and ^ in ca/id imagz ^o^unat. Howevet tki& 
output iilz may be /LC/ULLzed zitkoji on tapt, ok on d fASTRAND ^Itz, o/l phy^icalty cu pu^ichcd 
CjcuuU. Each oi tktst optioM U UJtiutnatexi in App V.Ua ion. tht llOi. 
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OUTPUT FILE |Tftt6 cteAc%cbeA thz ionn^at o< thz ""output iilt'' ^nom PATAREAP, ujhich i& uAcd 06 tht 
^pot i^oK AGGfOW. Whitz nonmaJUy wfUttzn on tapt, it may be on cjVu1&, and aticaij^ 
has cand imagt Honmat* ) 

In addition to printed output^ the program will produce output in punched card 
format for use with subsequent programs. The description below is intended to 
supply Information that will enable the Qrogranrner to utilize this output* 

The output file consists of a variable number of data sections for each 
questionnaire* Output for each section Is In tv^o parts: descriptive Information^ 
followed by the data* The data Itself Is simply the su.n of all the individual 
responses of data sets within the section. Both parts are variable length and 
format, but the logical record size Is always 80 characters (card Image)* 

In accessing the file. It is necessary to utilize Information provided in fixed 
format fields to determine the length and format of other Information* This allows 
for a variable amount of Information and an optimal data format* 

The first card In a section consists of the following: 

1) SIZE- the ""zMotmMtn 

3 digit number equal to the value of the size field on the header card; If 
there Is no header card or this field Is not specified, then the value is 
zero. 

2) NSETS- numbeA oi ""ntipondMU"", including ab^taintHA.) 

3 digits giving the number of data sets represented In this section* 

3) NQUEST- l^*^- nvaiAzK oi qut^tions on tht ^evoftt qut^tionnaiAZ.] 

2 digits giving the number of questions In the data* 



2 digits specifying the number of variable dimensions In 5) below* Maximum 
Is Ik. 



5) 01,02,. ..,01- 

variable list of dimensions giving the number of characters In each variable 
length field to> follow. 

The remaining cards preceeding the data contain pairs of fields, each having a 
label followed by Its Information content. There Is a fixed order of occurance, and 
the first four fields are always the same type. The first field Is given the label 
"QUESTIONNAIRE", and Its pair contains the 6-character Identifier supplied on the 

form command card* The third field contains "ID", and the fourth, the 10 of the 
section; or if no ID field was specified by. command cards, then this field Is filled 
Mith asterisks* 

If 10 subflelds were specified, they will appear next. In pairs of label 
content, followed by the variable fields from the header card !f any* 

Starting with the first position on the first card, the pairs of fields are 
located adjacent to each other until the length of a pair, added to the position of 
the previous pair would equal or exceed the 80-character limit. In the former case, 
the pair is simply located on the card, and the next pair If any will begin a new 
output card* If the latter occurs, then the pair Is located at the start of a new 
card; thus the fields never cross a card Image boundary. Output of variable fields 
Is terminated when the list of dimensions has been exhausted; output of section data 
begins with the next card. 



The data output for a section may be In one of three possible formats, depending 
on the number of data sets Included: 



NSETS'O^ no data output 

0<NSETS<10, 13 groups of six 1-dlglt numbers per card 

9<NSEI^^1004. 6 groups of six 2-dlglt numbers per card 

99<nSETS<IOO0, k groups of six 3-d I git numbers per card 



The six positions within a group represent the six possible responses to a given 
question on the questionnaire form. Each of the six digits gives the frequency with 
which that response was made for aM data sets In the section. Summing the six 
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digits within any group will produce the value of NSETS* 



OPERATION 



The program expects an input file as described In the section above. Multiolc 
questionnaires may^ be processed In a single run by follov/in;; the GND command card 
for the first questionnaire by only those command cards noccssary to specify 
parameter^ which differ in the next questionnaire. {6ZZ ZXOmptz in App. V.Ua.) 

If the first card after the START command for a given questionnaire Is 
recognized as a header^ then the program will expect all sections to be delimited by 
header cards^ and no automatic sorting by ID fields will be performed! Furthermore^ 
the 10 in the header will override ID's on all data sets up to the next header card! 
When a header card Is followed by another header card or an END command^ it will 
define a null section^ i.e.^ the descriptive information on the header will be 
transmitted to the output flle^ but v/lthout accompanying data. 

In order for sorting to be performed by the pro<;ram^ the first card following 
the START coininand must be a valid data card containing an ID. In the absence of 
header cards^ data sets will be grouped according to ID^ with extraneous invalid 
|D*s embedded \/ithin a group being overridden; the program will thus attempt to 
tolerate "bad" ID^s In the presence of a series of consistent ones. 

The less consistent the ID*s are^ the more groupings will occur^ with the "worst 
case" being all data sets grouped into separate sections. This will happen even If 
some of thein have identical ID's If they are not contiguous^ as the program will not 
remember back past a small number of groupings* 

If a header card interrupts sorting^ then all previous data sets are ln;nediately 
grouped, and che ID on the header card will establish a group which will include at 
least the next data set. 

'The intended use of this program is as a multiple pass operation. The first 
pass(es) are used to sort the Dijitek cards into sections and Indicate discrepancies 
in the data, and no output file Ts produced. After repairing bad data and Forcing 
correct groupings via header cards, a final pass is made to produce an output file 
for subsequent processing. 

IMP^)RTA^f^ - rfte pats hszk 6hould aJUo Ktad tht next ^ab^eation on p. 4.1/. 



IMPLEMENTATION 



The program consists entirely of code compatable with IBM FORTRAN IV level 
and Is run on a SYSTEM l^Q/kk. A s 1 Ightl y modlf I ed version Is available using 
UNI VAC 1108 FORTRAN V; In general^ the code Is machine Independent. 



INQUIRIES 



Further Information Is obtainable from the following source: 
Computer Systems Staff 

University of Maryland Cyclotron Laboratory 
Department of Physics and Astrcnomy 
College Park^ Maryland 20742 



* T/Ui it the, mode, in vohich PATS op^Aote^. 
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A.3>1 Further remarka on operation of the Program DATAREAD 

Section 4.3, together with the aastple run deck In Appendix D.l> should 
provide enough information for operation of DATAREAD. However the following 
points are worth mentioning* 

a) Output file . If the UNIVAC 1108 is being used for DATAREAD the most 
sensible procedure Is to produce the output file directly on tape. Alternatively 
the output can be punched as cards, or stored as a file on FASTRAND. The Imple* 
mentation of any of these choices Is shown In Appendix D.l. DATAI^EAD Is likely 
to be run several times before all data errors are corrected; only after that 
will the output file be ^*usefur\ It Is then used to produce an Initial 
printing for distribution of all sections using A6GF0RM1 (cf. Appendix F.l). 

Only then will certain "header^* information errors be discovered", e.g. wrong 
enrollment or instructor's name. These are most readily corrected using the 
1108 text editor on the previous output file - rather than by rerunning all of 
DATAREAD with corrected headers. To use the text editor the output file must 
be on FASTRAND ^ but It can readily be copied from tape at that time. Recall 
that a charge for permanant storage on FASTRAND Is levied by the C.S.C. 

b) The unit command cards (p 4.8 item 4). These show the historical 
origin of the program are being written for the IBM 360, In that **default** 
values correspond to that computer. A little thought will show that on the 1108 
If one wants the output file on tape ^ on a FASTRAND file, one can be somewhat 
cleverer than shown In Appendix D.la. For one can use the Internally set 
default value of unit 07 for this file, and use @USE 7»PUNCHTAPE or @USE 7,PTDATA 
without any unit card. However the listings In the appendix are more straight- 
forward conceptually. [See also section 5.12, p 5.23]. 

c) Program access (cf. Section 5.14, p 5.26). The program DATAREAD will 
normally be kept on a permanent tape. Appendix D.la shows how one would read 
the program off the tape (tape number P1361 say. In 8C9 format) and make It 
available for execution. 

d> Tape saver card . If one puts the output file on tape, and uses a 
SAVER tape - then It Is essential to supply a tape save card when submitting 
the run I 



4.4 A Technical Point on "Abstainers" > Also on "Enrollment". 

There Is one minor technical problem that should be mentioned at this 
point. As can be seen from Appendix E*l, AGGFORM (and also INITPRT) attempt 
to determine both the number of students vho "completed" the questionnaire, 
and the number who abstained (but participated to the extent of "rendering" 
an abstention; both of these to be distinguished from the number of students 
not participating at all (primarily those not present, but perhaps Including 
some students who left the classroom when they saw what was about to befall 
them!) . 

It had been hoped that the count of abstainers could be obtained directly 
from the questionnaire responses to question 1 (cf p. A1.2), but we have many 
cases (one being fully documented) of a student leaving question 1 blank (I.e. 
Indicating he wished to abstain) and then proceeding to fill In the remainder 
of the answer sheet! Presumably there will always be some students who In- 
sist on being perverse (but hopefully the large majority of students are 
basically willing to cooperate) • As can be seen from the Instructions to the 
students (p. Al.l), an abstainer Is requested to fill In the section ID num- 
ber, and leave everything else blank. As presently written, the program 
DATAREAD does not Initially select out such "blank" student data sets to 
determine the number of abstainers. All student answer sheets which are pro- 
cessed Into properly sequenced Dlgltek cards are Included In the summed dis- 
tributions and remain part of both the Input and output files of DATAREAD. 

The programs INITPRT and AGGFORM each estimate the number of abstainers 
are being given by the minimum number of "blank" responses to any of the 
questions on the questionnaire, for the summed set of answer sheets within 
one section. They then subtract this number of "estimated abstainers" from 
the number of "blank" responses to a given question to obtain the "corrected" 
number of such "blank" responses made to that question by non-total abstainers. 

All printed distributions are given with respect to the number of students 
estimated to be fully participating. Further AGGFORM actually alters the out- 
put file records for individual sections from the values inputted, so that 
the output file actually shows the "corrected" number of blank responses to a 
given question. [This is done in the part of AGGFORMl which is called TAPESR.] 

A similar "adjustment" is made in the printout, but not in the AGGFORM 
output file, if it is found that the number of student answer sheets processed 
is greater than the input value of "enrollment". Such a case (which does occur) 
does not cause any real difficulty. The number of answer sheets processed is 
clearly equal to the number of students participating, and this number is divided 
into an estimated number of abstainers and the number of non-abstainers as above. 
The statistical information at the beginning of each section printout (see e.g. 
p. Bl.l) is suitably modified to make clear what has occurred, and includes 
the quantity "percent of students participating who completed the questionnaire" 
instead of the usual quantity "percent of students enrolled who completed the 
questionnaire". [See also section 5.2. g, p. 5.3, particularly the footnote.] 

(The reader will now probably see the significance of some of the remarks 
made in section 3.5, particularly p. 3.6.) 



4.13 



It should be clear that the method presently used to estimate the number 
of abstainers is not truly satisfactory. It would be much better for DATAREAD 
to identify each abstaining student data set directly > and avoid the present 
estimation procedure. Such a programming modification should be straight- 
forward to implement (see Appendix J) . 

Alternatively (and perhaps more satisfactorily) the whole procedure for 
identifying abstainers should be reformulated. Both the instrucUons to 
abstainers and question 1 seem to miss their mark - at best they seem to 
encourage some students deliberately to "sabotage" the system. 

We should at this point reassure the reader.The proportion of students 
who attempt to defeat the system is very small, and the "errors" so introduced 
are not significant. The problem is not a basic one - yet it would be more 
elegant to solve it completely. 



5.1 



CHAPTER 5> AGGFORM EXECUTION MANUAL 
5.1 Introduction 

This chapter is a complete manual on how to run AGGFORM. Section 
5,2 simply summarizes the pertinent definitions and concepts which have 
been introduced in earlier chapters. Those who have read and itttderstood 
the previous portions of this report (in particular Chapter II) can skip 
this section. 

Section 5.3 gives a quick overall sketch of AGGFORM; Sections 5.4 
through 5.9 discuss various aspects of the program; and Sections 5.10 
and following get down to the details of preparing a deck of cards to 
run it. 

5>2 Summary of Definitions and Concepts 

a) Section 

A section refers to the smallest unit of students taught by one 
instructor , whether in lecture, recitation or lab; and whether one section 
of a course or a distinct course. 

b) Aggregate 

An aggregate is a set of percentage questionnaire responses 
usually formed by combining in various ways the percentage responses of 
two or more single sections. As an example, we could form '^Aggregate 
No. 100 - All Physics 10 Lectures'* by combining the single section responses 
for each of the Physics 10 lectures.* We compute the results in several 
ways. The first way is by weighting the number of responding students 
equally and the second is by weighting each section equally. (We sometimes refer 
to these as P-weighting & Q-weighting, leading to P- and Q-distributions. 1 

It is also possible to make aggregates of other aggregates. This 
leads us to a third type of weighting, that in which each such "input" 
aggregate is weighted equally. Since different aggregates can be made up 
of quite different numbers of sections, this is not the same as weighting 
by section. (We refer to thif latter type of weighting as R-weighting, 
leading to an R-distribution. ] 

Finally, it is possible to make aggregates out of a mixture of aggregates 
and sections. In this base the"weighting by aggregate" described in the previous 
paragraph becomes "weighting by component." This means that if we form an 
aggregate out of 3 aggregates and 2 sections, each of these 5 entities is 
called a component and is handled on an equal footing with each of the 
other components, regardless of whether it is a section or an aggregate. 
An illustrative example appears below under the entry for "component." [This is 
referred to as R-weighting^ leading to an R-distribution, since it is basically 
fhp flamft as the previous case.] 

* We use the "old" version of course numbers, and often fictitious ones 
at that. 



c) Combination 



If two instructors have shared teaching of a single section, 
the students will have filled out separate questionnaires for each instructor. 
Before these results can be aggregated with results from other sections, the 
two sets of questionnaires must be combined so that we will have just one 
set o£ responses for this one section. This is done by forming a special 
"combination" aggregate, in which the results from the two sets are simply 
averaged. (For all subsequent purposes, PATS considensuch a combination to 
be a single section) 

d) Complete Description 

Most T/A*s and some faculty teach more than one section of a 
given course. In this case we first form a "complete description" aggregate 
for each such instructor - namely the aggregate of all sections taught by 
that instructor. In the case of a T/A who only tead-.es one section, that 
section will in itself be the "complete description" for the T/A. It is 
these complete descriptions which are then used to form higher aggregates. 
Note that if one instructor teaches sections of two different courses, 
these should not be aggregated in this manner. Again we refer to the 
example below tmder "component." It will be seen that if we ' 

did not form such complete descriptions first, then we would have no way 
of weighting different instructors equally if some teach more sections than 
others. 



e) A Component 

This is a section or aggregate which is combined ("aggregated") 
on an equal footing with other sections or aggregates to form a new aggre- 
gate. Consider the following hypothetical situation for Physics 27 recita- 
tion sections: 



Section 


Taught By 


101 


Broim _ 


102 


Green ^ 


103 


Smith 


104 


Green 


105 


Jones ...y 


106- 


1 Smith 


107 


Goldman ^ 


108 


Jones w 




In this case Brown and Goldman teach one section each» while the 
others teach two sections each. First we form Aggregates No. 10 » 20 » 
and 30 (complete descriptions) for Green» Smithy and Jones « respec-* 
tlvely. Then Aggregate No. 20 for example » has two cotq)onents - sections 
103 and 106. In this case both con^onents are single sections. Now we 
form Aggregate No. A0» the aggregate of all Physics 27 recitation sections. 
This aggregate has five consonants - Aggregates No. 10 » 20 » and 30 » and 
sections 101 and 107. Two of the components are single sections; the other 
three are aggregates. Each io the "con5>lete description" of aiglven T/A - 
l.e.» it represents all sections taught by him. Thus all components are 
treated as **equals/* Note that section 108» for example^ is not a component 
of Aggregate No. AO; rather it is only part of one of the components 
(literally » a component of one of the components)* 

f) Eligible 

We define an Aggregate bv setting up a **test.** An eligible 
section or aggregate is one that passes this test. Thus to form the 
aggregate of all Physics* 29 lectures^ the test will demand that a pro- 
spective component be a Physics 29 lecture. Any section or aggregate 
that passes this test is now "eligible" for inclusion in the aggregate. 
Whether or not it is actually included depends on whether it passes the 
cutoff or not. 

g) Cutoffs 

There are three cutoffs In AGGFORM; we will call them Nos. 1» 
2» and 3. The particular numerical values used for each of these are set 
by the user in the AGGFORM control cards (cf Section 5.10). Each of these 
cutoffs refers to a particular criterion of response level. 

Consider an individual section > and the fraction f given by: 
(number of studen^rs who actually complete the questionnaire) divided by 
(the number of students enrolled*ln that section). Thus f « (responses)/ 
(enrolled). If this fraction is less then the value of cutoff #1, then this 
section is considered to be "rejected" > and will not be included in any 
aggregates unless forced in by the user. The results for this section 
will be printed out> but preceded by a very stem warning. 

If the section response level f is greater than or equal to the 
value of cutoff #1> but less than cutoff #2> the siection is " accepted" 
and is used in making aggregates; however the printed section results 
are preceded by a mild warning. Finally if the section response level 
is greater than or equal to cutoff #2> the section is again accepted and 
used in aggregates^ and the individual section results are printed without 
any warning. 

Cutoff #3 is the rejection level for aggregates and tests the fraction: 

(nutober of accepted components) 
g - _ 

(number of ab initio eligible components) 

*In the very special case alluded to near the end of Section 4.4(p4.11)> in which 
the nuinber of answer sheets processed is found to be greater than the number 
"enrolled" » the denominator is taken as the former number Instead of the latter. 
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If for any aggreRate^ this fraction "r" 
is less than cutoff #3, then the new aggregate is rejected and no results 
will be printed out for it. If the fraction of accepted components is 
equal to or greater than cutoff #3 but below cutoff #2 the results will 
be printed out with a warning. If the acceptance ratio is equal to or 
higher than cutoff #2, the results are simply printed out with no special 
statements. Thus there exists: a single section rejection cutoff, a 
warning cutoff for both sections and aggregates, and an aggregate rejection 
cutoff. At present these are set at cutoff #1 « .50, cutoff #2 « .70, 
cutoff #3 « .51. The use. of slightly different rejection cutoffs .50 
and .51 for sections and for aggregates is imposed m the basis of 

the consideration that a response of exactly one-half the students in a 
sec^ ion was felt to be acceptable; however, for aggregates this was not 
felt to be the case, particularly^ in the obvioiis case of an aggregate of 
two sections, which would simply be identical to one of the sections if- 
the other was rejected.** [These cut off values .50, .70 and .51 are actually 
internally set as "default" values, and will be used automatically unless 
other value? are read in via the print control card (cf Section 5.10.3a,p.S.20) J 
^ 5>3 Preparing to Run AGGFORM 

AGGFORM is a program which can make up to 20 aggregates at a time 
(i.e., in a single execution); control cards furnished by the user direct 
the program as to how many aggregates to make, what to include in them, 
etc. The program produces printed output as specified by the user, and 
writes an output tape containing both the contents of the original input, 
aid the newly-formed aggregates.* This output tape can then be the input 
for another execution of the program, and so on until all desired aggre- 
gates have been formed; and at. the end of a given execution, the output 
tape will contain all aggregates formed in all executions up to that point 
(as well as the original input). Each execution can thus build on the re- 
sults of previous executions. Since any number of executions can be 
included in one con?>uter run, the entire processing can in principle be 
completed in a single run. In practice, however, the processing is done 
in stages to check for and correct errors in the single section data and 
in the control cards. 

AGGFORM is actually two different programs: AGGFORMl, which reads 
in the original single section input (on tape or cards); and AGGF0RM2, 
which reads as input the tape produced by the previous execution. It 
is this second part which is repeated as many times as is necessary. 

In order to run the program, it is necessary to go through the steps 
listed below. An explanation of each step will be found in the remaining 
sections of each chapter. 



It is important to remember that at the end of each stage of running AGGFORM, 
the output tape contains everything that has been done up to that point, 
including the original input. Thus, for example, printout for the original 
single sections could just as easily be obtained at the end of the process 
as at the beginning. 

**It is possible to change the cutoff valties between executions of AGGFORM. 
Such changes will not affect ^.ggregates already coiq)uted, but will be 
implemented for all new aggregates. 
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TABLE 5.1 Sequence of Operations Needed to Run AGGFORM 



!• Decide which aggregates are to be formed, and what each is to be 
called. A **tltle" of up to 80 characters, including blanks, must 
be specified for each aggregate. (Any number of additional comment 
cards (within reason) may be added if desired.) 

2. Assign a number from 1 to 9999 to each aggregate, remembering that 
the final printout will be in order of increasing aggregate number. 
It is wise to leave room in the numbering system for later insertions. 
This can be achieved by numbering by tens; for example, aggregates 
10, 20, 30 and 40. Aggregates may be created in any order, subject 
only to the restriction that components of an aggregate must be 
created in executions whidi are prior to the execution in ^ich the 
current aggregate is being formed. It is convenient to use 4 digit 
nuid>ers with the first digit representing the questionnaire number. 

3* Decide how many copies of each aggregate will be needed for distri- 
bution to instructors. ("Multiple Mania**). This number must be 
assigned when the aggregate Is first created; it cannot be changed later 
(unless the program is altered). 

4. Determine which aggregates will be combinations, which will be complete 
(Ascriptions, and which will require weighting by component. 

5. Divide the list of aggregates to be made into groups of twenty or less, 
each group to be formed in a single execution. Remember that combin- 
ations and complete descriptions must be created in an early execution, 
to be available as components of new aggregates in later executions. 
For the sake of economy, as few executions as possible should be made. 
But to allow for later "corrections", try to plan on fewer than 20 
aggregates per execution. 



I 
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6. For each execution, make up the contrul' cards which direct the program. 
There are basically three kinds of control cards, for three parts of 
the program : TAPE, SELECT, and BL0C1. These cards are described in 
secticm 5. 10 below, (p. 5.12) 

7. Decide on the type of printout desired for each execution. Printout 
is saved in the computer's mass storage; this way additional copies 
can be generated as desired. 

8. Get the program intp the storage banks of the computer. It normally 
resides on a magnetic tape on a shelf at the Computer Science Center; 
there is also a back-*q> tape. Before the program can be executed, it 
must be loaded into the computer. The procedures needed are described 
in Section 5. 14 below (p. 5.26). 

9. Put the control cards together with the standard run decks (see listing 
in Appendix f). Supply appropriate tape numbers (e.g., tape with single 
section Input on it, tapes for output of successive executions). 

10. Submit the job. 
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5. A Which Aggregates Should be Made? 

All "combination" and "complete description" aggregates should always 
be made. Which higher aggregates are made is a matter of choice. 

In general, all other aggregates will combine at least three individual 
sections. [To aggregate two sections U and V does not make it any 
easier to compare U with V, and the aggregate (IH-V> will seen to have 
more normative value "than is truly appropriate.] 

Initial aggregates should be chosen to provide meaningful combinations 
.equivalent or "peer group" sections. E.g., "all sections of c -rse X ." 
or all first year physics major courses," etc. One is then likely co >o 
on to higher level" aggregates, which contain within them previous aggre- 
gates, e.g. "all first and second year physics major courses;" and then even 
all undergraduate physics major courses," and perhaps even higher. In the 
following we consider specific examples v ich will denonstrate both the 
straight-forward nature of most aggregates, and the complexity required to 
handle unusual situations. 

An easy way to indicate the content of each aggregate is to draw a 

heirarchial tree, ' using the header card deck printout (section 3.6. D, p. 3.12). 

In appendix CI we show such a sample heirarchical tree, and in apppndix 

C2 we give the corresponding list of names and comments. [ The numbers in 

the rlghthand margin are the nimiber of copies needed in Multiplemania,cf Section 
j.o. 2, p. 5. 8. J ' 

Ut us now look at the hierarchical tree and headers in oetail 
(Appendix CI, p. Cl.l). On the first page of the tree, there is little 
that is not self-evident. Since each "section" represents a separate faculty 
lecture group, only weightings by student and by section are used. In gen- 
eral, aggregates contain at least 3 sections, except for 3110, which corres- 
ponds to a separate subsequence of courses. The reader should thoroughly 
understand what is happening on this first page before attempting the' more 
unusual situations treated in the next two paragraphs. 

On the second page of the tree, things become quite complicated. 
Notice first the use of comments when appropriate. Next, consider the 
6000 sequence of aggregates.*. Course 429 is actually the s:aae course as 
621, so that 429010162 is really the same section as 621010162, but con- 
tains responses from different students. (Undergraduates enroll under 429, 
graduates under 621.) Aggregate 6010 puts these responses together, and 
leads to a "complete description" of Martin. Unfortunately, it is counted 
as two sections [AGGFORM as yet has no facility to add parts of one section 
and produce one section, but such a modification is fairly easy to program.] 
Actually Martin, Zom and Pechacek taught separate groups of students, and 
each should be counted equally, and each as one section. Aggregate 6030 
assembles these three sections into one. But since aggregate 6010 and 
6020 are each counted erroneously as 2 sections, weigjiting by sections is 
not correct. Hence one uses the weighting by coa^onent. This weighting 
treats aggregate 6010, aggregate 6020, and section 62101061 Zom equally. 



* This case is sufficiently unusual and con5>lex that the average reader need 
not be overly concerned with it. However, it does provide useful insight 
into the handling of sudi troublesome situations. 
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A ••conunent" is included in the "description" of Aggregate 6030 to warn about 
the incorrectness of the weighting by section. 

Let us then turn to the 7000 series. In this case, as shoim on the 
"comments," sections are team taugjht. Hence one first makes individual 
combination sections. Aggregates 7000 to 7030 and 7100 to 7120. Each of 
these count as one section. From these, one then makes course aggregates 
7060 and 7160, in the regular way. Finally an aggregate over all such 
courses is performed to obtain 7200. But one also wishes to obtain complete 
descriptions of each professor. This is done on a separate tree with aggre- 
gates 7040, 7050, 7140 and 7150. 

The above examples should be sufficient to indicate the types of 
flexibility available. 



5.5 Aggregate Nunbers 

Every aggregate miist be assigned a distinct number between 1 and 9999. 
The numbering scheme need not use every integer, so that one can arrange 
the numbering in a convenient way. We have, for example, divided up the ^ 
numbers so that numbers from lOOO to 1999 correspond to aggregates involvxng 
questionnaire 1. Those from 2000 to 2999 involve questionnaire 2. Within 
each of these groups we number by 10* s, thus allowing room for later in- 
sertions. An important function of aggregate numbers is to determine the 
order of the final printout, lowest numbers being printed first. Since 
aggregates which we want to see printed side by side may actually be 
created in separate executions of the program, this feature is a very 
necessary one. 



5.6 Printed Output 

This section is a brief discussion of what is actually printed for 
each section or aggregate, and how much control the user has over which 
aggregates or sections are to be printed at a time. 

The computer can produce up to five different "batches" of printed output 
at a time (i.e., in one execution of the program). A given batch will consist 
of a table of contents which lists the sections or aggregates which are in- 
cluded in that batch. This is followed by the output for each section or 
aggregate included in this batch; and finally there is a statement that the 
end of this batch has been readied. The user can control to a certain extent 
what will appear in each batch. Within a given batch, aggregates will be 
printed out in order of increasing aggregate number, and single sections 
in the order in which they were read in. 

The form of the output for each single section or aggregate is as follows. 
There is first a title line and TpTace for comments. Then comes a summary 
of the number of students enrolled, number of responses, etc. For aggre- 
gates there is a list of all included coii?)onents, followed by a list of all 
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rejected components. Then the distributions for each question are printed 
with headings to explain their meaning. 

The different types of output available in a given execution are as 
follows (see explanation following the list): 

1. Newly Formed Aggregates 



2. Multiple Copies of all Aggregates 

3* All Aggregates, one copy each; or sll complete descriptions 

4. All Aggregates in a given interval of aggregate number 

5* All single sections 



5>6.1 Newly Formed Aggregates : 

The first kind of output refers to the fact that processing the data 
requires several executions of the program. "Newly formed aggregates" 
simply means those aggregates formed in the most recent execution. This 
is the kind of output one wants when running each execution for the first 
time. Output of previotis executions will already have been checked for 
errors and corrected, so only the new output will need checking. After 
all errors have been caught, one will probably not want to generate this 
type of printout anymore* 

5>6.2 Multiple Copies of all Aggregates : 

Multiple Mania, as it is commonly called, is necessary if one wishes to 
distribute copies of the aggregates to the individual instructors in- 
cluded in them (sending each instructor a complete copy of all sections and 
aggregates would be too massive an undertaking, and might additicially 
dissuade an instructor from poring over the results). A complete descrip- 
tion for one T/A involves just that T/A, whereas the aggregate of all recita- 
tion sections in Physics 10 and 11 involves perhaps 20 T/A*s. If we want 
to send each instructor a copy' of every aggregate in which he is involved, 
we obviously need different numbers of copies for different aggregates. 
Multiple Mania allows the user to specify exactly how many copies he wants 
of each aggregate.* This batch will then have, say, one copy each of 
Aggregates 10, 20, and 30, followed by five copies of Aggregate 40 (which 
might include the previous aggregate plus two other sections), and so on. 
This batch of output can then be separated by hand for distribution to 
individuals. (There is an option which allows one to obtain Multiple 
Mania for only a specified interval of aggregate numbers. See comments 
below on the fourth kind of output.) 

5.6.3 All Aggregates; 

The third type of output is self-evplanatory. It is intended for public 
display, for the department chairman, etc., usually together with a copy 
of the single, sections produced by the fifth type of output. (There is an 



With the program as presently written, this number must be specified when 
the aggregate is first created and cannot be changed iacer. It is the 
number on the right in Appendix C2. 
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option on this type which, if asked for, produces print just for all com- 
plete descriptions.) 

5.6.4 All Aggregates in an Interval ; 

The fourth kind of output has not proven as useful as was first hoped. 
The idea was to print all aggregates (one copy each) within a given interval 
of aggretate numbers, say everything from aggregate number 10 tp aggregate 
number 90. In practice there is not much need for such an option, except in 
cases where there is a need for extra copies of certain aggregates for 
special pruposes, which one does not wish to meet by altering Multiple Mania. 

5.6.5 All Single Sections ;* 

This is basically self-explanatory. A possibly useful alteration in the 
program would be to allow printing of just some of the single sections as 
opposed to all or none. 

It is important to understand that, except for the "new aggregates" 
batch, the program stores all printed output in the computer's mass storage 
facilities. Then additional copies of each batch may be easily generated. 
Thus if one needs four listings of all aggregates, it is not necessary to 
run the program four times. Rather one generates the initial listing once 
(in the final execution) and then makes three additional copies. For this 
reason the numbers of copies requested in Multiple Mania should be divided 
by the least common denominator of all such numbers. (Making Multiple Mania 
twice as long is much more costly than making two copies of a shorter version.) 



5.7 Type Numbers, And All That 
5.7.1 Type Number 

This is a number assigned to every aggregate or single section which desig- 
nates what "kind" it is. In particular, it provides a way in which 
the user can inform the program about aggregates which require special 
treatment. For example, calling a certain aggregate Type 1 tells the 
program that this is a combination and should be handled accordingly. 
Such information must be given to the program through the type nuinber. 
The following list gives those values of the type number which the user 
may set; there are other values used internally by" the program, but these 
need not concern us here. Note that a blank (or zero) is equivalent to 
Type 5. Since this is the most usual case, type number will normally not 
need to be 3pe cifie d. However one must spec ify type 1 & type 4 aggregates. * 



Type Number > Meaning 

0 (or blank ) Same as 5 

1. This aggregate is a combination 

4. This aggregate is a complete 

description 

5. Normal Aggregate 



*For the record, type 2 & type 3 refer respectively to single sections & aggre- 
gates which are (sub) components of complete descriptions, & type 4 also refeis to 
single sections. See also Faction 5. 10. 3b, p. 5. 22. 



'^^ fQ^^^ ^ aggregate to be accepted (I.e., to be printed out and to 
be available for inclusion in higher aggregates), one adds 10 to the type 
number.* This is necessary when an aggregate would normally be rejected 
due to insufficient response and for some reason we do not wish to lose it. 
Suppose we are making^^^complete description for Mr. X, a T/A who teaches 
four recitation sections. We see that two of his sections had very low 
response, but the other two had very good response. When the program checks 
out this aggregate, it will therefore find that exactly 50% of the eligible 
components were accepted. If the cutoff #3 is set at 50% (as is now the 
case), this aggregate will be rejected. But we may decide that it is worth- 
while to print out this particular aggregate, even though half the sections 
are rejected. We can force it to be accepted by calling it Type 14. That 
is, we call it Type 4 to signify that it is a complete description, then add 
10 to the type number to force it acceptable. 

Whenever an aggregate is rejected (i.e., fails to meet the cutoff) the 
program automatically sets its type number negative. Thus, for example, 
type numbers 4, -4 and 14 all refer to complete descriptions, but one is 
normal (accepted), one is rejected, and one is force accepted. As will be 
more apparent later, it is hence convenient also to define a " reduced type 
number" . 

This is basically the same as the type number, except that it makes no dis- 
tinction between normal, rejected and forced accepted values. Thus the reduced 
type number for a complete description is always 4*. [a technical point ab^llt 
reduced type numbers is discussed in Appendix H.5.2,pH.5,2]. 



5.7.2 Prof - T/A Number ^ 

The Prof-T/A number is a code which indicates the* rank* of the in- 
structor for a given section (or for an aggregate if it is a "complete 
description"). The values of this number are as follows: 

T/A Prof 

Number Meaning 

■'■ " ^ professor (set at 1 i f only one prof 

for the section, otherwise used 
sequentially 1,2,3,4 to distinguish 
profs) 

^ " ^ T/A (set at 5 for one T/A, otherwise 

used sequentially 5,6,7,8,9 to 
distinguish T/As ) 

5.7.3 Course Category ; 

Course Category is used basically to distinguish honors from non-honors 
courses and Physics from Astronomy courses. The values and code are given 
below. (As presently programmed, this concept does not apply for aggregates) 

*Thus one enters tvoe number 11.. 14., or 10. to fore*> an ;icrcrrpp;»fp which -fs 
combination, a complete description, or "normal", respectively, cf Section 
5.10.3.b, p. 5.22. 



Course Category Meaning 

0 regular physics 

1 "honors" physics 

2-4 for future physics use 

5 regular astronomy 

6-9 future astronomy use 



5>8 Tlie Specification of Aggregates 

We direct the program by first setting up a series of tests by use of 
one group of control cards (SELECT cards). Then in a later group (BL0C1 
control cards), we tell the program what to include in a given aggregate 
simply by telling it which test to use. For example, to obtain an aggre- 
gate of all Physics 10 and 11 lectures, we first set up a test to be used 
by the program as it examines all prospective components (i.e., all single 
sections and all aggregates which have been formed in previous executions.) 
In this particular example the test will check to see if the course number 
is equal to 10 or 11. If the prospective components include both Physics 10 
lectures and Physics 10 recitations, we must also test to see that we are 
indeed dealing with a lecture. We give this total test a number say 10. 
Then ]ater in the control cards, when we give the aggregate a number and 
name, we tell the program that anything that passes test //lO is eligible 
for inclusion in this aggregate. 

How one "defines" an aggregate is critically important with respect to 
the question of weighting by component. First we must emphasize that the 
term "weighting by component" applies only when the components are aggre- 
gates or a mixture of aggregates and single sections. If the components 
are all single sections, then weighting by component does not exist as a 
separate method of weighting. In this case it is identical to weighting by 
section. 

An example should make clear that the way in which we define an aggre- 
gate determines whether weighting by component will be produced or not. We 
can form the aggregate of all^' Physics 15 and 16 lectures in two ways. We 
can either abinitio aggregate the three Physics 15 lectures and the four Physics 
16 lectures, in which case we have seven components, and weighting by component would 
be equivalent to weighting by section, (and hence it is not separately computed); or 
we can aggregate two components formed in a previous execution - the aggre- 
gate of all Physics 15 lectures, and the aggregate of all Physics 16 lectures. 
In the latter case we do get weighting by component. Note that this kind 
of weighting always involves aggregating components which have been pro- 
duced in previous executions. 

It is thus seen that a heirarchical tree (see section 5.4 ) does not 
give full information on the definition of an aggregate. One must really, 
specify the components. Generally it should be clear which weightings are 
appropriate, and hence at what level the components should be taken. For 
faculty, PATS primarily uses weighting by section. 
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5.9 Override Controls 
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There are several override controls in the program which allow the 
user to alter the normal course of action. In particular, they allow 
him to overrule the results of the cutoff tests. One of these controls - 
the ability to force an entire aggregate to be considered acceptable whe- 
ther it passes the cutoff or not - has already been discusserin section 
5.7. This is accomplished by adding 10 to the type number. 

Another control allows the user to force into a given aggregate a 
component which would normally not be included due to insufficient response 
Forcing in a component is handled by using the tests exactly like those 
used to define the aggregates themselves. 

For example, we set up test #1 so that each component that passes 
this test will be included in a certain aggregate, if it meets the cutoff 
test. Suppose we know that three eligible components will probably fail 
the cutoff, but we want one of them forced in whether it fails or not We 
then set up another test, say test //2, to look for this particular com- 
ponent. Then on the control card which tells the program to make the 
aggregate, we say include everything that passes test //I if and only if it 
also passes the cutoff, but separately"also"include anythKg that passes 
test if2 no matter what. The way in which this is actually set up will 

mSr^K'SS'ci? I2gll§!lli ^t4fie^Aa!lS«^5 ^^'^^i"" "i^l "^^^ included 
5. in Making Up the Control Cards 

We stress once again that care and patience are essential in settinz 
up the control cards for AGGFORM. As with any computer program, simule 
mistakes (such as the omission of a decimal point) can cause major errors. 
The entire deck of cards which runs AGGFORM will be discussed later At 
this point we are concerned with just one portion of that deck, namely the 
cards which AGGFORM itself will read in order to direct its operations, 
ft Nearly all of these control cards follow the same format. The 80 columns 
of the IBM card are treated as 8 separate "fields" of 10 columns each. [When 
one key punches, a useful aid is to use a "drum card" which should have the form 
lAAAAAAAAAlAAAAAAAAAl...etc.]. Each field may contain a number with an explicit 
decimal point, or a set of "alphanumeric" characters. In the following, the 
reader should take careful note of the colvimn structure, and the spaces be- 
tween "items" on each line (i.e. on each IBM card). In our examples we often 
show the field division columns before 1. and 10/11, at 20/21 etc. and sometimes 
also in small circled type the actual column nunibers. ("Word" entries must 

a lways start at the beginning of a field, though numbers wi tb decimals need not.) 
The first control card read by the program is a title card for the 

ITfc^V^ Ji V ^'^"'^'^ °^ ^^^^ P^8« °f °"tput). This 

IS to3 lowed by three separate groups of control cards, for three parts of 
the program - TAPE, SELECT, and BL0C1. TAPE reads the input. SeLecT sets 
hi rnTi^'Z . "^^''^ describe each group of components which will then 

the c5nto.t aggregate. BL0C1 makes the aggregates and controls 

the printout. Note that the SELECT tests are set up separately from the 
^l!f ^.!„i ^"""^^^ aggregates; only in these latter cards is 

l^l lUr"", told which test goes with which aggregate. [This separation of 
the tests from the control cards which name the aggregates is slightly 
awkward, but results from AGGFORM is being embedded within the structure of 
the general summarizing program SUMX. The advantages of using SUMX were 
SUMX ]° "''^"'^^^ disadvantages . See Md. Tech. Rpt. #763 for more on 
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The first card of each of these groups of control cards simply bears 
the name of the group (e.g. TAPE). The first group (for TAPE) contains only 
one or two additional cards; the other two, however, can have varying 
numbers of cards, and thus each must end with an end card - END SELECT or 
END BL0C1, .respectively. The latter is followed by a card that says END 
PASS. Thus the overall set-up is as follows: 



(title card) 
TAPE 

(1 or 2 cards) 
SELECT 

(many select cards) 
END SELECT 

^many bl^cl cards) 
END BL0C1 
END PASS 

Except for title cardo, all control cards are divided into eight 
fields of 10 columns eaclu Each field may contain one number or one or 
more words, or may be left blank. If a word is to be placed in a particular 
field, it must begin in the first column belonging to that field. It is 
convenient to treat numbers the same way, though it is not necessary. 
Thus if the number -26. is to be placed in the third field of a particular 
card, the minus sign would be placed in column 21. m numbers must have 
decimal points. 

5.10.1 Tape Cards 

As mentioned before, there are actually two separate programs which make up 
AGGFORM. These are called AGGFORMl and AGGF0RM2. The former is used 
only on the first execution, the latter for all subsequent executions. 
The principal difference between them is that AGGFORMl reads the original 
single section data, which is cither on cards or on a magnetic tape, and 
has one particular format; whereas AGGF0RM2 reads the output of a 
previous execution, which is on magnetic tape in quite a different format. 
For this reason, the TAPE control cards are different for the two programs. 
We discuss the two programs in turn. 

a) AGGFORMl 

The first card simply has the word TAPE (beginning in column 
one). The second and third cards are currently as follows: 

-99. p. . j: 

2I3,I2/13X,I1,7X,I9,6X,I3,7X,I3,15X,I1,7X,I1/10X,4A6,7X,6A6/5A6,A4 

These tell the program exactly how to read in the single section 
data. If the form of the single section data is ever altered (e.g. 
the course number is shifted to a new position on one of the cards), 
then these two cards must be altered appropriately. 



5,U 



b) AGGF«JRM2 

The first card still contains the word TAPE, The second card 
has numbers in the first two fields; these two nuinbers are the 
identifying label for the tape being read (i^e,, the output tape 
of the previous execution), [This previous execution will have 
been directed to write the proper ID on the tape via one of the 
BL(iCl control cards. See discussion p. 5,21] It is customary 
to let the first number be the year of the survey in question and 
the second be 1., 2., or 3., to signify Spring, Summer or Fall, 
Thus for Fall 1971 we have 



TAPE 
1971. 



5, 10. 2 SELECT 



3. 



Up to 20 "tests" are defined by the SELECT control cards. Each 
one must be given a number between one and twenty, and they must be 
set up i n order of increasing test number ; %owever^ it is permissible 
to skip numbers. That is, a set of tests numbered 1,3,5,7,9,11 arid 
13 would meet these requirements. 

All prospective components which pass a given test are then 
eligible for later being made into an aggregate by BL0C1, and will 
be included if they pass the cutoff criteria. Suppose we wish to form 
an aggregate of all physics 10 and 11 lectures except honors sections. 
We use SELECT caids to set up a test that the input sections musn pass 
in order to be included. In this case, the test would be that the 
section in question have its course number equal to 10 or 11, and that 
the instructor is a faculty member (so we do not include recitations 
or labs), and that the section is not an honors section. Set up using 
SELECT cards, this appears as follows: 

^ f ■ 

AND ITEST 

INTERVAL -6. 10. U. 

AND ITEST 

INTERVAL -22. 1. 3. 



AND ITEST 
NTEFVAL -32. 0. 0. 



The numeral "1" in the word "ITEST" signifies that this is test 
number one. The "AND" cards take the place of the underlined "and^s" 
in the above sentence description of the test with one extra "and" at 
the begin^ing^ The "INTERVAL" card means that the quantity indicated 

^Contrast this with the statement in Section 5. 10. 3. b, p. 5. 22 that sets 
of aggregate control cards may be assembled in any order! 



by the code in the second field ("-6" means the course number, 
-22 means the TA-PR0F number, » "-32" means course category)* must 
lie in the interval specified by the last two numbers. In the case 
of the course number, above, it must be greater than or equal to 
10, and less than or equal to 11. If we wanted just Physics 10 
lectures, then we would set the last number equal to 10 and have a 
single-valued "interval." The course category equals one for physics 
honors sections, so the third INTERVAL card specifies non-honors 
sections. If our data only includes faculty, then there is no need 
to distinguish faculty from TA's and the second INTERVAL card (witJL 
its AND card) is unnecessary. 

Siq)pose we wanted two intervals of course numbers included in 
our aggregate. Then our course SELECT cards might look like this: 
® ^ 



-6. 



AND ITEST 
INTERVAL 
9R , 
INTERVAL i-6 



; 10. 



30. 



11. 



32. 



This says that a section will pass the test if its course number 
Jies in the interval 10 to 11, or in the interval 30 to 32. The 
"0R" card itself is optional and may be left out. Thus, the following 
is equivalent to the above: 



iAND ITEST 



'INTERVAL 



INTERVAL -6. 



-6. 



10. 



30. 



11. 



32. 



In the examples to follow we will leave out the 0R cards, but the 
effect is the same as if they were there. 

Suppose now we wish to make three aggregates, for Physics 10 
Lectures, Physics 11 Lectures, and Physics 10 and 11 Lectures com- 
bined. For simplicity we will deal just with course numbers and 
ignore the TA-Prof number and course category. We could write our 
tests as follows: 



* See Section 5.7 for explanation of type no., TP-Prof no., and course 
category, etc. 
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^i) @ 
AND ITEST '^ 

INTERVAL -6. 10. 
AND 2TEST 

INTERVAL 11. 
AND 3TEST 
TEST 1. 
TEST 2. 

The third test says that a section passes this test if it 
passes either Test 1 or Test 2. Use of this device saves a lot of 
redundant writing when the tests involved are longer than one or two 
lines each. [It can also introduce redundancy if, for example, tests 
1 and 2 have an element in common, such as the requirement that the 
instructor be a faculty member, not a TA., but such duplication is 
harmless. A section may pass more than one eligibility test, but it is 
only included once in any case, within one aggregate.! 

As illustrated in the example above, the end of one test is 
signified by the beginning of the next one. Note that there is always 
one (and only one) blank column after the word "AND" on the AND card. 
There is no blank between the test number and the word TEST. Thus 
double-digit test numbers give us "AND 12TEST", etc. 

It is important to note that the test number specified on a 
TEST card must be lower than the number of the test being made. 
Thus the following sequence is illegal : 

AND 2TEST 

TEST 3. 

This is also true of the NOT TEST instruction illast rated below. 

Finally, suppose we have set up one test to make an aggregate, 
but wish to exclude sections passing this test from some other aggregate. 
Then we proceed as in the following example (the code "-7" is for the 
section number):* 




i 



■11. 



I 

I 
I 





* "Section number" here means the number assigned to a particular 
section of a course in the official schedule of classes. This is 
the "classical" use of the term "section", as opposed to the more 
general usage elsewhere in this report. 



AND STESXi 
INTERVAL 1-6. 
INTERVAL 1-6. 
AND 8TEST 
INTERVAL i-7. 

I 

AND 9TEST i 
INTERVAL -6. 



AND 9TEST 



N0T TEST 



8. 



■30. 
,32. 

ilOO. 

30. 



30. 
32. 

100. 

32. 



Again we have dealt just with course numbers for simplicity. 
Test 8 picks up Physics 30 section 100 and Physics 32 section 100. 
Let us suppose that these are the honors sections, there being no 
honors section for Physics 31. Then Test 9 gets us all the non- 
honors sections for Physics 30, 31, and 32; that is, it picks up 
all sections with a course number in the interval 30 to 32, except 
those specified in Test 8. 



By this time the reader may have observed that a lot of the 
testing can be done in more than one way. In the example above we 
got the honors sections by referring to the specific section numbers. 
We could also have tested the course category to see if it were equal 
to one. Or to specify that a certain aggregate should include faculty, 
not TA's, we can either test directly on the TA~Prof number or on 
the questionnaire number, since there are usually different question- 
naires for faculty (lectures) and TA*s (recitations and laboratories). 

One should understand clearly the relation between OR*s and 
AND's . Suppose we set up a test to ask for either Course 10 0R 
Course 11 AND section 100. To be accepted, a prospective component 
must be section 100 and it must be course 10 or course 11. One might 
be tempted to read the test as written above to say that it requires 
either course 10 (any section) or section 100 of course 11. This is 

The following is a list of the quantities usually tested and the 
codes for each: 
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QUANTITY 


CODE 


Questionnaire No. 


-4. 


i^ourse Numoer 


-6. 


Section Number 


-7. 


First digit of course no. 


-23. 


First digit of (3 digit) section no. 


-24. 


TA - Prof. Number (cf Section 5.7) 


-22. 


Type Number " " " " 


-26. 


Reduced Type Number " 


-33. 


Aggregate Number 


-25. 


Course Category (not for aggregates)* 


-32. 


Instructor's Name (first 6 letters) 


1008. 


Instructor's Name (next 6 letters) 


1009. 



To form the aggregate of all sections taught by a given TA, say Mr. 
Casey Jones, we look for his name, testing six characters (including blanks) 
at a time. The codes are given above; the six characters are put in the 
fifth field of the INTERVAL -ard (col. 41-46). Thus we might have 

p < W 

llNTERVAL ,1008. jjONES 

It is important that the name be given exactly as it appears in the 
rllviu ^^^^ the header cards). If the name is given as "JONES, 

CASEY then the first six characters include the comma. Usually the first six 
characters are quite sufficient to identify a name unambiguously. Suppose, 
however, we have a Mr. K. Johnson and a Mr. S. D. Johnson; the names will 
probably be listed in the data as "Johnson, K." and "Johnson, S. D.". Then the 
tests would be as follows: 



D m 

AND ITEST^ 
INTERVAL 1008. 
AND ITEST 
INTERVAL 1 1009. 



AND 2TEST 
INTERVAL 
AND 2TEST 
INTERVAL 



1008. 



1009. 



J0HNS0 
N, K. 

J0HNS0 
N, S. 



[Notice that periods, commas and spaces all count as letters for the purpose 

tfr bi''rLr^^^^"^''f ^^"^ letters". The computer compares charac- 

ter by character, and has no imagination. ] 

*As presently progranmed, the course category value is not preserved for aggregates 
even when applicable. Hence one can only test on course category for individual 
input sections. 



I 
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I It is important that the tests be set up so that they unambiguously 

define what is to be included in a given aggregate. For instancy, a test 

I designed to include Physics 10, Section 109 would pick up both the recitation 
section iH09f and the laboratory section //109; these can be distinguished by 
adding a test on questionnaire no. 

y An aggregate continues to have, say, a course number as long as all its 

i components are all from the same course. As soon as an aggregate contains compo- 
nents from more than one course, it ceases to have a course number. This same 

J procedure applies to all identifying quantities. t Thus, in the example given 
on pp. 5.2-5.3 Aggregate 10 still has the appropriate course number and question- 
naire number, has no section number, and has an instructor's name. It is a 
^ complete description. To form Aggregate 40 one wishes to aggregate the compo- 
I nents: aggregates 10> 20, 30 and sections lOl^and 107. The easiest way to get 
this (ignoring questionnaire number, etc, ) is 





* 




AITO 7TEST 


40. 




INTERVAL 


-6. 


27. 


AND 7TEST 






INTERVAL 


-33. 


4. 



We are aggregating all sections and aggregates having the correct course number 
(code -6.) for Physics 27, which are also cotiplete descriptions. (Code -33. selects 
reduced type number, which is 4. for a complete description.) Notice that sections 
101 and 107 are indeed complete descriptions. 



p The program automatically sets all input sections as being complete descrip- 

I tions (type number 4.) when they are first read in. When in an execution some 

such input sections are put together in an aggregate^ which is itself specified 
_ to be a complete description, the section type and reduced type niimbers are changed, 
I so that subsequently only the aggregate Xs considered a complete description. On 
i the other hand, if the aggregate is not specified to be a complete description, 

the sections each retain their original reduced type number 4. (though the type 
r number may change as an indication of the accept, reject or force-accept status; 
I hence always use reduced type number for testing). 

P Thus in the example on pp. 5.2-5.3 and above, each of the eight sections 101- 

I 108 are initially tagged as a coTi?)lete description. In the execution in which 

*• aggregates 10, 20 and 30 are made, those aggregates must be specified to be complete 

^ descriptions. In that cat;e, after that execution of the sections, only 101 and 107 

r will retain their complete description status and have reduced type number 4.. 

f Aggregate 40 must be made in a subsequent execution, and will then be correctly 

assembled. [If made in the same or an earlier execution, it would be incorrectly 
I* computed — but its content list i^ould then provide information which would corres- 
£ pond to the error that had been ir-^de.] To form Aggregate 40 one might then test on 

r educed type number and course and questionnaire number. 



f 

r 



As a brief remark, we note that if one wishes to keep track of which test goes 
with which aggregate, he could put the aggregate number (with decimal point as always) 
in the second field of the first "AND" card of the test. This will have no affect on 
the program, but can serve as a useful reminder to the user. We always do this in 
practice, as indicated in this example for aggregate 40. 

tA present programming "defect" prevents this from being true for course category. 
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5aO>3, BL0C1 



There are several different kinds of control cards for BL0C1 — the "print" 
control card and the printout title cards; and the sets of Aggregate control 
cards. 

5>10>3>a> Print Control Card . 

The print control card has the following information: 
Field Meaning 

1 Print Control Number (see below) 

2 Kinlmum Agg. No. 

3 Maximum Agg. No. 

(The second and third fields are used only if printout 
for a specific interval oi Agg. Numbers has been re- 
quested. Otherwise they are left blank.) 



A First ID word to be put on output tape 

(usually Vear of survey; see TAPE cards, above) 

5 Second ID word for the output tape 
(usually code to distinguish semester) 

6 Cutoff level #1 for single sections^ These specific values are 
(currently .50) internally set as "default" 

values, and will be used 

7 Cutoff level #3 for Aggregates > if the relevant field is 
(currently .51) left blank; but an entry 

In the field overrides 

8 Warning cutoff #2 the preset value, 
(currently .70) j 

The print control number itself is a five digit number. 
Each digit corresponds to one type of printout (ste section 5.6); 
if the digit is zero, no printout of that type is obtained. For 
some of the digits, the particular value of the digit specifies 
one of several options available. The scheme is as follows 
(digits are numbered one to five going from left to right); 
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Print Control Number 

Digit Value Meaning 



1 Print out all aggregates formed in this 
execution, except combinations 

2 Print all new aggregates, including 
combinations 

3 Print all new combinations 



1 Print Multiple Mania 

2 Print Multiple Mania for aggregates in 
specified interval 



1 Print all aggregates including combinations 

2 Print all complete descriptions [ (reduced) 
type 4 aggregates, and those single sections 
x^hich are complete descriptions!. 

1 Print all aggregates irfiose aggregate 

numbers lie in the specified interval 



5 1 Print all single sections 

The print card must be followed by one title card for each 
type of printout selected; i.e., the number of title cards Siiit 
equal the number of non-zero digits in the print control number. 
These cards may be (and frequently are) left blank. They must be 
listed in the order of the (non-zero) digits of the print control no. 

Consider the following example: 



20001. 



NEW AGGREGATES 
SINGLE SECTION INPUT 



rfA) p fe> IsD 



The print control number, 20001., calls for two kinds of 
printout, and there are correspondingly two title cards follow- 
ing the print card. The second and third fields of the print 
card are mused. The tape ID will be 1971/3 (for Fall, 1971) 
The cutoffs are .5, .51, and .7, respectively. 
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5>10>3>b> Aggregate Control Cards 

The next cards after the title cards are the Aggregate Control 
cards. For each aggregate there is a card with the word AGGREGATE 
in the first field, a title card, and up to lA additional comment 
cards (if desired). The first card contains the following: 

Field Cols. Contains 

1 1-10 AGGREGATE 

2 11-20 Aggregate Number 

3 21-30 Type No., if necessary » (not the reduced type no.) 
A 31-AO Test Number 

5 Al-50 Override Test Number, if needed. 

6 51-60 Nimiber of comment cards 

7 61-70 pQR number- normally left blank (see App. H.l) 

8 71-80 Nimiber of copies of this aggregate 

needed for Multiple Mania. 

Again, all numbers must have decimal points. Any field not needed 
for a given aggregate can be left blank. Following this AGGREGATE 
card, as stated above, are the title card for that aggregate (up to 72 
characters, including blanks) and its comment cards, if any.* An example 
follows: 

^ 1/ <a) 

AGGREGATE .1010. [l. p. :2. 

PHYSICS 37 - TWEEDLEDUM AND TWEEDLEDEE 1 * 

These are the cards for aggregate no. 1010. The type nuiAer is 
one, so it is a combination. Any section that passes test number 2 
will be included, but only if it meets the cutoff (since there is no 
override in effect). There are no coaanent cards, and 2 copies are 
needed for distribution. 

The set of aggregate control cards for any particular aggregate 
must of course be in the sequence AGGREGATE card, title card, and com- 
ment cards. But these sets can be entered in any order. There is no 
need for these to be in ascending order of aggregate number. 

*AGGFORM automatically inserts to»o comment cards for any "combination" aggre- 
gate (cf Section 5.2c,p.5.2). These state "This aggregate is treated as a 
simple section. The students enrolled, etc. are taken as the average for the 
included sections". Naturally "extra" comment cards may also be inserted: 
cf Appendix F.2, p. F4, Aggregate 7000 for such an example. 
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5.11 Cards verstts Teletype 

Once the control cards have been prepared, the run deck can be made up. 
Before this discussion .however, a few comments are in order concerning the 
computer. This program was written for the University of Maryland 1108 
(UNIVAC) and the following sections apply specifically to this computer. 

Up to now we have t;ilked about preparing IBM cards, which would be 
put together into a run deck and submitted to the computer. This is called 
a batch run — one submits a deck of cards with the various instructions 
for the computer and the control cards for the job, and one gets back, after 
the job has been run, this deck of cards plus any printed output produced by 
the job. If, say, one of the control cards is in error, one punches a new 
card, replaces the erroneous one with it, and resubmits the job. 

There is another way to proceed - namely, by using the teletype. Here 
one has direct access to the computer, and no cards are involved. Just as 
one can prepare the control cards and have them ready for insertion into the 
run deck, so one can type in the control card images with the teletype and 
store them m the computer's mass storage until needed. If an error in a 
control card is found, one can correct it from the teletype very easily. 
There is a (per dien) charge for use of the 1108' s mass storage, but the 
convenience may well be worth the expense (at the most $2.00 per day). 

Even though the authors' personal choice would be to proceed by using 
the teletype, the following sections are presented in terms of a batch run. 
This IS easier to explain, and involves the introduction of fewer additional 
complications (like the EDITOR program used to correct card images stored in 
the computer). Anyone familiar. with teletype operation of the 1108 will have 
no difficulty translating our discussion to that frame of reference 



5.12 Some 1108 Terminology 

Even if the teletype is not used, some understanding of the computer's 
mass storage and the Exec 8 control language is prerequisite to the success- 
ful utilization of AGGF0RM. Some of the statements below are not str^ctW 
Sif?f simplification; however, the .reader should not encounter any 

difficulties due to this. ' 

1)^ A "file" is the standard unit of storage on the 1108's mass 
storage; it s size is defined merely by how much it contains. There is al- 
ways an upper limit on how much a particular file can hold. This limit is 
set by the computer and is usually quite amplef* If it is not, the user can 

nhJ.? of f ""/n"^' f ^" "^'^^"^ ^he file. Once a file has been 

obtained (and caj^glogued" - see below) it will be kept on hand by the 1108 
T^Jk cS!^^ . ^^l V^'^hout being used. After that it may no longer be 
saved. One can also obtain a temporary file which is good only for one com- 
puter run; as soo n as the run is over, the file is destroyed. 

*We oversimplify, and often refer to this mass storage as FASTRAND. 

**At present this limit is 250 tracks (or granules) if the file is to be saved . 

cf Appendix F, particularly F.5. 

***This number has currently been changed to 20 days—it clearly may change.' 
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Files have names assigned to them by the user. A file has one permanent 
name, the one given to it when first obtained by the user, but it can have 
other (temporary) names. The name can be made of letters, or numbers, or a 
cjombination of both, as long as the total is 12 characters or fewer. 

Three types of files which can be used in AGGF0RM are program files, 
data files, and print files; they differ, as indicated by their names, in 
their contents. 

A file is available for use when it is "assigned" to the run (job). When 
a file is no longer needed, one can then "free" it (i.e., "un-assign" it). 
When one frees a temporary (non-catalogued) file, however, it is destroyed. 
One destroys a catalogued file by "deleting" it. For the 1108, this sequence 
appears as follows : 

@CAT,P PRTFILE, F/1//250 

@ASG,AX PRTFILE 

@FREE PRTFILE ' 

@DELETE PRTFILE 

Here we have catalogued, assigned, freed, and finally deleted a file named 
PRTFILE. Each line consists of the "@" in column one, an instruction field, 
then at least one blank, and then the file name. The field at the end on the 
(aCAT card allows one to obtain a larger file than the computer usually allows. 
For Multiple Mania, which requires a very large file, the 250 should be changed 
to at least 500. It will be observed in what follows that soinetimes file names 
are followed by periods. The period is usually optional, but sometimes it is 
necessary (and in at least one case it Is not allowed!). 

To give a file a temporary name, we employ the @USE statement, as follows: 
@USE 13, PRTFILE 

This says that if there is a file named PRTFILE assigned to the tun, it can 
also be referred to as "13". Thus, if this file has been catalogued, we 
could have the following: 

@ASG,AX PRTFILE 
@USE 13, PRTFILE 

@FREE 13 - 

When the last statement above has been executed, the file named PRTFILE will 
have^ been freed. Let us now suppose that this file is a print file containing 
the "printed" output of the program, namely one of the five kinds discussed in 
Section 5.6. To transfer this information to paper, i.e., to actually print it, 
we use the following command: 

@SYM,U PRTFILE,, PR2 



'TIOTE: The CAT and ASG can be replaced here by the single statement @ASG,CPX 
PRTnLE,F/l//250. This assigns the file and catalogues it if and only 
if the job terminates correctly* 
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This will only work if the file in question is a catalogued file and Is 
not assigned to the run when the @SYM command is given. That is, the file must 
be freed prior to printing. Repetition of this command produces additional 
copies of the output, one copy for each occurrence of the command. 

The above instruction directs the printing to occur on printer //2 (PR2); 
other choices are PRl, PR3, and PR9. PRl and PR2 are reasonably reliable and 
produce easy-to-read output, but are slow. PR3 is very fast but does not pro- 
duce beautiful print; it should be used to print large batches of output, or 
anytime when the quality of the print is not essential. PR9 is currently un- 
reliable. Since any one of these printers could be shut down for repairs at 
any time, it is wise to check with the 1108 dispatcher before submitting the 
job. A @SYM command to an inoperative printer will simply be ignored by the 
computer, and no print out made. 

Our final comments in this section concern magnetic tapes.* Tapes must 
also be "assigned" to the run so that they will be available to the program. 
The required statement is as follows: 

@ASG , T INTAPE , 8C , 9 876N 

The "T" stands for temporary; as soon as the run is over, the tape will 
be removed from the computer (if it hasn't already been freed and removed). 
"INTAPE" is the name given to the tape while it is assigned to the run. 9876 
(say) is the permanent identifying number of the tape, and the "N" specifies 
that we only want to read from this tape — we do not wish to write on it. 
[If we want to write on the tape we use the letter "R" instead of "N".] When 
we are finished with the tape, we simply say: 

@FREE INTAPE 

As with the files, the (§USE statement supplies a teti?)orary name for the 
tapes; for example: 

0USE 9, INTAPE 

which gives the temporary name 9 to the tape which is actually tape 9876. 



5.13 Specific Tapes and Files for AGGF0RM 

AGGF0RM always expects to read from and write onto tapes and files having 
certain sp«jcifir numbers as their names. AGGF0RM2, for instance, expects the in- 
put tape to be tiamed 9. This can be achieved either by assigning the tape 
with the name 9 to begin with, or by assigning it with another name and exnploy 
ing the @USE statement to give it the temporary name 9. For tapes, the latter 
procedure is unnecessarily indirect; for^print files, however, it can be very 
useful. 

First we give a complete list of the file (unit) numbers involved: 



*We do not attempt to be complete on this subject. Tapes must be SAVED by use 
of "tape save" cards, and maintained - or they may be destroyed ! SAVER tapes 
are destroyed after 48 hours unless a tape save card is submitted. 



Aggform File Numbers ^ 25 

Output tape (AGGF0RM1 & 2) 11. 

Input tape* (AGGF0RM2)* 9. 

Multiple Mania Print File 13. 

All Aggregates Print File 15. 

Interval of Aggregates Print 

File 21. 

Single Sections Print File 23. 

The "New Aggregates" output is printed along with the regular computer 
output for the job, and hence, no special provision need be made by the user. 
When, however, the user requests other types of printed output (i.e., non- 
zero digits in the Print Control Number), he must provide a file with the appro- 
priate number onto which the program can write. Otherwise catastrophe results. 

In general one can simply assign (and catalog) files with the appropriate 
number. The exception to this occurs when one wishes two different copies of 
the same type of printout. Suppose the user wishes to process data from two 
different "groups" in the same week, e.g. faculty and T/As, and have, say, the 
list of All Aggregates from each data set stored on the computer at the same 
time. Needless to say, cannot have two distinct files with the same name 
(in this case 15). Rather, one must catalog the files with distinct names, 
then give each the temporary name 15 only for the duration of the appropriate 
AGGF0RM execution. 

One final comment about the QSYK statement: Print Files are identified 
by the number of the job (number on job card) during which they were written . 
This enables the (?rYM'ed output, which is printed separately from the rest of 
the job, to be returned to the user with his normal output. But this means 
that if on Monday I write on file 13 in job #2000, and on Tuesday submit job 
#3000 which does nothing but SYM out an extra copy of file 13, this new copy 
will be labeled with the number 2000 and will be stored on the shelves of the 
Conq>uter Center unde/ that number, separate from my deck of cards and normal 
output from job #3000. Hence it is essential to keep a record of the job num- 
bers in which print files are first created!t 
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5>1A Obtaining the Program 

It is intended that the program should reside on a magnetic tape (with 
perhaps also one or more backup tapes containing additional copies ) and be 
read into the computer only when execution is desired. Let us assume the 
appropriate tape number is XXXX; oae prt|>ares for execution as follows: 



Input to AGGF0RM1 is handled in a special way, even if it is on a tape. 
See listing of runs in Appendix F. 

A truly skilled operator can change this job number on the file to a 6 
character name before it is symmed. He would first copy the complete file 
(or else make sure that a copy had been rolled out the previous night in 
case of catastrophe), then from TTY use ^FILEDIT <file name> and A 1,28 
and then C 5,'KACSER* to change the fifth word (which is the job number) to 
e.g. KACSER (no more than 6 characters). Similarly , of course, an AGGFORM 
run originally submitted on reletype should always be given a distinctive 
run identifier. 



QASCT 



P,8C,XXXXN 



@CAT,P PATSY., F/1//250 

(aC0PIN p., PATSY 

@FREE P 

Actually there are two possible formats for writing on tape. The 8C 
in the first card refers to one of these. The other format is 8C9. The 
tape must be read in the same format in which it was originally written. 
Hence 8C should be replaced by 8C9 if appropriate. 

The run decks in Appendix F assume that the program has already 

been read into a file called KACSER*PATSY. + Normally one intends to run at 
least several executions over a course of a week or so, and it makes sense 
to leave the program stored in the computer for the duration. This is why 
PATSY was catalogued in the above example. If one did not need the program 
again, he could DELETE the file at the end of the job, or, alternatively, 
merely assign it as a temporary file in the first place: 

@ASG,T P, 8C,XXXXN 

@ASG,T PATSY, F/1//250 

(aC0PIN P. , PATSY 

@FREE P 

Then when the job is finished, PATSY will vanish. 



5.15 Putting Together the Entire Run Deck 

Appendix F contains saii5>le decks for running AGGF0RM1 and 

AGGF0RM2. The preceding sections should have given the reader sufficient 
understanding to allow him to follow the exanq)les and run AGGF0J8M correctly. 
Details of the @RUN card can be found in the 1108 EXEC 8 USER'S MANUAL. 

The exan5)les should be studied carefully together with the material in 
this chapter. They have "editorial" explanations, and refer to the heirarchy 
of aggregates discussed in Appendix C. 



t The "qualifier" KACSER* is the name of the individual who first 
created the file PATSY in a run which had hid name as qualifier 
(in this case Kacser). 
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APPENDIX A. THE QUESTIONNAIRES 

A.l. The Seven Questionnaires Actually Used Fall 1971 

UNIVERSITY OF MARYLAND 

Physics Department 
Teaching Questionnaire 
Form 1 - "General" Courses (Lecturer) 



op'in?ra\^\t^h^'1hysiL'\°"d1s: '° ^"""^^^ 
nation wfirbe uLf ir^L ways ""t^i^:"'"' ^^^^ 

promotion. evaluation of faculty with respect to retention and 

course (which should L on th" blackbird) iff^ f^/he code number for this 
Ing in the appropriate blocks e f r j „ \ ^ ""^ "<"='<- 

Your instruc^L ^ill'^eJ^-htUb ia'ie ° :,S"f°t' 
after rl"'/""'?" coULta, " 



NUMBER 



General Instructions 



1. 



Of VaTibSx'i 

> •>■ -v X ' 



2. 



All entries must be on the STANDARD ANSWER SHEET 
Comments must be written on the back of this sheet. 

either^'^ne'b'/ T'l' ^"'^^ ''"^^"^ ^^^^ have 

either one block only or no block marked. Multiple 

answers are noL allowed, so that a wrong answer I 

must be erased carefully and completely. To mark 

iTzziir '^'^^ ^^^"^ ^^^"^ 

Nrr'^^r^o^rstL^:^^^^^^^^ -Bit space labelled^Student 

blackboard. ' "^^ mrormation should also be on the 



^SSpIration. Mh^^isc blank answer sheet. Thank you for yiSF^ 



II 



Questions; 



1. Vfith regard to this questionnaire: 
(a) 
(b) 



(f) 



You basically approve of such questionnaires, and hence you are re- 
spondlng to the vhole questionnaire. 

You find such questionnaires an Imposition, but In spite of th^s vlev 
you are responding to the vhole questionnaire. * 
t\>ll\JL^^^^ ^^^"^^ disapprove of such questionnaires, and hence 
(PLEASE, after filling m the code number) Intend to abstain completely. 



r 

2. Tlie lecturer presented the historical origins of ideas and concepts 

(a) most of the time 

(b) some of the time 

(c) very infrequently 

(d) never 

(£) no answer (i.e. leave blank) 



3. In reference to question 2, I prefer - 

(a) Much more 

(b) more 

(c) about right 

(d) less 

(e) much less 

(f) no answer 



4. The lecturer related physics to other disciplines - 

(a) very infrequently 

(b) some of the time 

(c) never 

(d) most of the time 

(f) no answer (i.e. leave blank) 

5. In reference to question 4, I prefer - 

(a) much more 

(b) more 

(c) about right 

(d) less 

(e) iBjch less — 

(f) no answer 

6. The lecturer brought out the "philosophical" beauty in the stbjcct, i.e. 
some feeling for the internal symmetry and logic of the subject - 

(a) most of the time 

(b) some of the time 

(c) very infrequently 

(d) never 

(f) rto answer (I.e. leave blank) 
/. In reference to question 6, I prefer - 

(a) much more 

(b) more 

(c) about right 

(d) less 

(e) much less 

(f) no answer 

B. The lecturer related class topics t., students » lives and concern, and 
more generally to tiie outside world - 

Ca) a lot of the time 

(b) very infrequently 

(c) some of the time 

(d) never 

(e) as often as possible 

(f) no answer 



A1.3 



1 



9, In reference to question 8, I prefer - 

(a) nuch more 

(b) nore 

(c) about right 

(d) less 

(e) much less 

(f) no answer 

0. Due to my taking this course, my interest and appreciation for the subject - 

(a) have increased greatly 

(b) have increased a bit 

(c) have remained unchanged 

(d) have decreased somewhat 

(e) have been completely turned off 

(f) no answer 



11. 



(a} a great deal 

(b) quite a lot 

(c) a little 
(d} not at all 

(e) had negative effect 

(f) no answer 

12. I evaluate the lecturer 'rknowledge of the subject as being - 

Ml ... W satisfactory 

d) less than satisfactory incompetent 

(f) no answer ^ 

13. His ability to get the material across to me is - 

(a) excellent (b) good (c) satisfactory 

(d) poor (e) zero (f) no answer 



Ik. Rid preparation for class is 

a) excellent (b) good (c) satlsftictory (d) less than satisfactory 
e) poor (f) no answer 

15. The lecturer - 

(a) usually started and ended his class on time 

(b) sometimes started late 0£ ended late 

(c) frequently started late or ended late 

(d) frequently started late and also ran over 
(c) often called off class or^ didn't show itp 
(f) no answer 

16. Ibe course as a whole was planned and organized 

(a) extremely well 

(b) well 

(c) acceptably 

(d) rather poorly 

(e) extremely poorly 

(f) no answer 

17. In my opinion the inclusion in the course of lecture demonstrations, 
films » and visual aids was - 

(a) too much 

(b) good 

(c) adequate 

(d) not enough 

(e) too little 



18. The lecturer^s apparent concern in teaching this class showed that he was - 

(a) exceedingly interested (b) iisually quite interested 

(c) fairly interested (d) only occasionally interested 

(e) bored (f) no answer 

19. His attitude toward questions in class was - 

(a) very encouraging (b) usually encouraging (c) adequatelv receptive 

W flippant (e) discouraging (f) I have no' opinion 



20. From my experience, the teacheris availability for questions outside 
of class hours was - 

(a) encouraging" and generous of his time 

(b) usually available 

(c) barely adequate 

(d) discouraging and usually unavailable 

(e) always unavailable 

(f) I have no opinion 

21* The feeling between tne lecturer and the students was * 

(a) that of strong goodwill (b) that of gpo^will (c) neutral 

(d) negative (e) antagonistic (f) do answer 

22. Hie exams - 

(a) always covered material or techniques emphasized in the course 

(b) usually covered material or techniques emphasized in the course 

(c) sometimes covered material or techniques emphasized in the course 

(d) frequently covered material not emphasized in the course 

(e) were totally unrelated to the course 

(f) no answer 

The exams and grading in the course were - 

(a) much too easy; easy to get good grade without adequate understanding 

(b) a little too easy 

(c) very fair 

(d) somewhat too difficult to get a good gra/le 

(e) much too hard 

(f) no answer 

24. The assignments in the course were - 

(a) much too hard or much too long 

(b) generally somewhat too difficult or somewhat too long 

(c) about right for the course 

(d) not quite adequate 

(e) non-existent or grossly inadequate 

(f) no answer 

25. The problems and/or questions assigned were - 

(a) very useful in understanding the material 

(b) helpful in understanding the material 

(c) somewhat helpful in understanding the material 

(d) generally not well selected 

(e) poorly selected 

(f) no answer 

26. The text(s), handouts, readings, etc. were - 



(a) excellent (b) good (c) satisfactory 

(d) less than satisfactory (e) poor (f} no answer 



A1.5 



1.5 



lIviTirSllS Jm! •pproxiwtely the smc 

itvti le Kiich this person w«s teaching, you would - 

W taterly seek his section out 

m taka his section if convenient 

it) have AO stroni feelin|t$ 

W mU his section if easily possible 

M avoirf hia like the plague 



2t. tha Mis of this person's teaching In this course, I think that he • 

(a) aeriu appropriate recognition for his truly outstanding teaching 

d) is • food teacher 

(c) has the potential of beconlng a good teacher 

(d) Is aa adequate teacher, but without Ruch potential 
(t) should not be retained as • teacher in this course 
(f) no answer 

2t« Tkm B8th*aad/or physics background that I assiMed was needed in this 
caurse was - 

(s) sufficient to handle the course 

^) inadequate, but the instructor filled in the g^s 

(c) inadequat<r and further, the instructor failed to fill In the gaps 

{€) greater than needed 

(•) poorly related to the course 

(f) ao aaswer 



50« IB opiaioR the ^nfral level and speed of the course were - 

(a) such too herd tnd/or auch too fast 

C^) soaeiAat too hard and/or somewhat too fast 

(c) Just right 

(d) e little too easy and/or a little too slow 
(•) aoch too easy and trivial 

(f) tto Mswer * 

SI. lis Qsa of oxaaples and illustrations was - 



(a) excellent and effective (b) good 

(c) adequate and helpful (d) less than satisfactory 

(e) aoticaably lacking (f) ao answer 

for aadi of the ftoliowing possible bad habits, stark 'block fa) if and only If 
tha lastructor suffei-s froa it, and it would be desirable for hin to correct 
it; otherwise aaka no aark - 

52. iaaudible ~' 

SS. writing was illagible 

54. spoke either too fast or too slow 

55. aonotonous style of speaking 

56. poor organization on blackboard 

57. lectured to tha blackboard 
5t« paced too auch 
59. Ikuas" and *1iaws» 
40. distracting or nervous wmnerlsns 
41 caustic or sarcastic aanner 
42. any other bad habits? Hark (a) if 

**ftt** and specify on the reverse • 
side of answer sheet 

For each of tha following positive qualities, aark block (a) If the Instructor 
has It to a high degree - 

45. Is articulata (a) yes 
44. has excellent organimion on blackboard (a) yes 
M» has very well paced lecture (a) y^a 

46. has a lively style of speaking (a) yas 

47. has cheerful or pleasant disposition (a) yes 
46. spaaks directly to the class (a) yes 

49. has a sense of huaor when called for (a), yes 

50. he Botivated ne to do ay best work (a) X^s 

51. be discussed his actions, decisions, 

and Selection of topics (a) yes 

52. he aade difficult topics easier to 

understand (a) yts 



(a) should be corrected 

(a) should be corrected 

(a) should be corrected 

(a) should be corrected 

(a) should be corrected 

(a) should be corrected 

(a) should be corrected 

(a) should be corrected 

(a) should be corrected 

(a) should be corrected 

W yes 



A1.6 



S4 • »"^'i«<> Mjor points (,) yes 

54. he stated objectives for ereh class ' ^ 

section . . 

5«/^«'"»«'»P"«i"l .ppiictions. . Ill 

Mark block (a) if yes, ,nd specify on (a) yes 

the reverse side of answer sheet ^ 
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57. 



I 



ERIC 



58. 



' °" «ld* of answer she 

U) yea 



<•) (f) ao (I.e. leave blank) 

S9. Your class is - 

J*5 sophomore (c) junior 

(d) senior (e) graduate or special student (f) I prefer not to answer 



60. 



6K 



62. 



63. 



Your major is in the area of - ^ 

^aI IIJJU??!!?" 1 sciences (c) life sciences 

(d) physical sciences (e) other (mark here, and specify on the reverse 
side of answer sheet) (f) I prefer not to answer reverse 

How aany hours of work outside class did you put in on this course? 
^ej^lude hours related specifically to lab) - 

(a) between 0 and 2 hours per week 

(b) between 2 and 4 hours per week 

(c) between 4 and 6 hours per week 

(d) between 6 and 8 hours per week 

(e) over $ hours per week 

(f) I prefer not to answer 

In the course do you expect to get - 

(a) a good grade (b) an average grade 

(0 a poor grade (j) I prefer'nof to answer 

How Many classes of this instructor have your missed . 

W none (b) very few . (c) few 

W) ■»ny (e) nearly all fA f^. 



(e) nearly all ' (f) j prefer not to answer 

You are taking this course - 

W to fulfill a General Education requirement 

Cb) to fulfill a requirement of my program 

(c) elective, chosen because of my interest in the subject 

(d) elective, chosen because I needed an easy course 

m ? J h«w<* the course or professor was good 

(f) I prefer not to answer * 

65- Of the courses that would have served my purpose, this one was - 

(a) only choice that fulfilled my needs 

(b) my first choice 

(c) ay second choice 

(d) only course still c^en 

(f) I prefer not to answer (i.e* leave blank) 



UNIVERSITY OF MARYLAND 

Physics Department 
Teaching Questionnaire 



Form 2 - "Service" ':ourses (Lecturer) 



; r 



[I 

M 

ERIC 



The purpose of this questionnaire is to obtain in an anonymous fashion your 
opinion about the physics and astronomy teachers and courses. This infor- 
nation will be used in many ways — to improve each teacher's performance, 
to help the chairman assign faculty members to courses, and to provide 
important input in the evaluation of faculty with respect to retention and 
pronotion. 

Do not put your student number in the box labelled student nuober in the lower 
right hand comer of your answer sheet. Instead put the code njimber for this 
course (which should be on the blackboard) in that nine digit space by block- 
ing in the appropriate blocks, e.g. Code Number 012210021 would become- 
Your instructor will get the tabulated results of this 



questionnaire, together with any written comments, 
•fter the final grades in this course have been 
submitted. 



General Instructions 



NUMR(.R 



■ ~"n r-T-T-l — I— 



Li.rrm 



»- ill 



1. 



3. 



4. 



11 



All entries must be on the STANDARD ANSWER SHEET. 
Comments must be written on the back of this sheet. 

Use a Number 2 pencil only . Each question must have 
either one block only or no block marked. Multiple 
answers are not allowed, so that a wrong answer 
must be erased carefully and completely. To mark 
the response (f) leave all spaces blank. This will 
be counted. 

Black in your course code number in the nine digit space labelled"Student 
Number' on your Standard Answer Sheet. 

Finally, please write on the top of the Standard Answer Sheet the course 
number; section number; name of the specific person whom you are evaluating; 
and whether this is a lab, recitation, or lecture; (e.g., "Phys 100. 
Section 201, John Doe,. Lab.") This information should also be on the 
blackboard. 



Even if you do not fill out the questi o nnaire please carry out instruct ions 3 

52lA. and return the otherwise blank answer sheet. Thank you for your 

cooperation. - ; 



II 



Al,8 



2.2 

Questions 

1. With regard to this questionnaire: 

(a) You basically approve of such questionnaires, and hence you are 
'^^^Po^ding to the whole questionnaire. 

(b) You find such questionnaires and Imposition, but In spite of this 
view, you are responding to the whole questionnaire, 

inf^!cE^''"^^ disapprove of such questionnaires, and 

JompIeJeS ""^"^ '"'""^ 

2. I evaluate the lecturer's knowledge of the subject as being - 

(a) excellent (b) good (c) satisfactory 

(dj less than satisfactory (e) incompetent 
(f] no answer 

3* His ability to get the material across to ne is - 

(a) excellent (b) good (c) satisfactory 

t^J Poo'^ (e) zero (f) no answer 

4. His preparation for class is - 

Ca) poor ^ (b) satisfactory (c) good 

(d) less than satisfactory (e) tecelleat (0 no answer 

5. The lecturer - 

(a) usually started and ended his class on time 

(b) sometimes started late or ended late 

(c) frequently started late or ended late 

(d) frequently started late and also ran over 

(e) often calls off class or doesn't show up 

(f) no answer 

6. The course as a whole was planned and organized - 

(a) extremely well (b) well (c) acceptably 

(d) rather poorly (e) extremely poorly (f) no answer 

7. Injny opinion the inclusion in the course of lecture demonstrations 
films, and visual aids was - ' 

(a) too much (b) good * (c) adequate 

(d) not enough (e) too little (f) no answer 

8. The lecturer - 

(a) seldom made the underlying physical ideas clear 

(b) never made the underlying physical ideas clear 

(c) usually made the underlying physical ideas clear. 

(d) always made the underlying physical ideas clear 
(f) no answer (i.e. leave blank) 

9* The lecturer's ability to make the subject interesting - • 

(a) excellent (b) good (c) satisfactory 

Poo'^ (e) zero (f) no answer 



10. 



Tlie lecturer's apparent concern in teaching this class showed that he was - 

(a) exceedingly interested 

(b) usually quite interested 

(c) tdirly interested 

(d) only occasionally interested 

(e) bored 

(f) no answer 
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II. His attitude toward questions in class was - 

(a) ver>- encouraging (b) usually encouraging (c) adequately receptive 
(d) flippant (e) discouraging (f) I have no opinion 

12^ From ny experience, the teacher's availability for questions outside 
of class hours was - 

(a) encouraging and generous of his time 

(b) usually available 

(c) barely adequate 

(d) discouraging and usually unavailable 

(e) always unavailable 

(f) I have no opinion 

J13. The feeling between the lecturer and the students was - 

(a) that of strong good nill (b) that of good will 
(c) nevtral (j) negative 

(e) antagonistic ff) no answer 

U. Docs the lecturer make the distinction between the important points of the 
subject and incidental infoiroation - 

(a) yes. he always makes that distinction very clearly 

(b) he makes that distinction clear in most instances 

(c) sometimes he does, sometimes he doesn't 

(d) for the most part such a distinction is not made 

(e) never 

(f) no answer 

15. The exams - 



(a) 
(b) 
(c) 
(d) 



always covered material or techniques emphasized in the course 
usually covered material or techniques emphasized in the course 
sometimes covered material or techniques emphasized in the course 
frequently covered material not emphasized in the course 

(e) were totally unrelated to the course 

(f) no answer 



16. The exams and grading in the course were - 

(a) much too easy; easy to get good grade without adwjuate understanding 

(b) a little too easy ^ ^ 

(c) very fair 

(d) somewhat too difficult to get a good grade 

(e) much too hard 

(f) no answer 



17. The assignments in the course were - 

(a) muc*- too hard or much too long 

(b) generally somewhat too difficult or somewhat too lone 
(cj about right for the course 

(d) not quite .adequate 

(e) non-existent or grossly inadequate 

(f) no answer 

IB. The problems and/or questions assigned were - 

(a) very helpful in understanding the material 

(b) helpful in understanding the material 

(c) somewhat helpful in understanding the material 

(d) generally not well selected 

(e) poorly selected 

(f) DO answer 



W# The text(s), handouts, readings, etc. 



were 



■ - . satisfactory 

(d) less than satisfactory (c) poor (f) no answer 



20, The grading and conBentary on the homework was - 



(a) carefully done and helpful 

fo) adequately done 

(c) less than satisfactory 

(d) arbitrary and unfair 

(e) not done 

(f) no answer 

21. If you were to take another physics course at approximately the same 
level in which this person were teaching, you would - 

(a) eagerly seek his section out 

(b) take his section if convenient 

(c) have no strong feelings 

(d) avoid his section if easily possiblt? 

(e) avoid him like the plague 

(f) no answer 



22. 



On tne basis of this person's teaching in this course 1 think that he - 

£) n'i'goS't"S« '^i- outstanding teaching 

(c) has the potential of becoming a good teacher 

fl »^ *<J«luate teacher, but without much potential 

(c) should not be retained as a teacher in this course 

(rj no answer ^ 



23. The math and/or physics background implied by the prerequisite of this 
course was - 



(«) 

(b) 
(c) 
(d) 
(e) 
(f) 



sufficient to handle the course 

inadequate, but the instructor filled in *:he gaps 

inadequate and further, the instructor failed to fill in the gaps 

greater th^ needed 

poorly related to the coarse 

1 don't know the official prerequisite 



24. In my opinion the general level and speed of the course were - 

(a) much too hard and/or much too fast 

Cb) somewhat too hard and/or somewhat too fast 

(c) just right 

(d) a little too easy and/or a little too slow 

(e) much too easy and trivial 

(f) no answer 

25. His use of examples and illustrations was - 

(a) excellent and effective 

(b) good 

(c) adequate and helpful 

(d) less than satisfactory 

(e) noticeably lacking 

(f) no answer 

For each of the following possible bad habits, mark block (a) if and only if 
the instructor suffers irom it, and it would be desirable for him to correct 
it; otherwise make no mark - 



26f inaudible (a) 

27f writing was illegible (a) 

28. spoke either too fast or too slow (a) 

29. monotonous style of s/eaking (a) 

30. poor organization on blackboard (a) 

31. lectured to the blackboard (a) 

32. paced too much (a) 

33. »*hums'* and "haws'* (a) 
3A. distracting or nervous mannerisms (a) 

35. caustic or sarcastic manner (a) 

36. any other bad habits? Mark (a) if 
"yes" and specify on the reverse (a) yes 
side of answer sheet 



should be corrected 
should be corrected 
should be corrected 
should be corrected 
should be corrected 
should be corrected 
should be corrected 
should be corrected 
should be corrected 
should be corrected 



For each of the following positive qualities, mark block (a) if the instnictor 
has it to a high degree - 



37- is articulate 

38- has excellent organization on blackboard 
has very well paced lectures 

AO. has a lively style of speaking 

Al. has cheerful or pleasant disposition 

A2. speaks directly to the class 

A3, has a sense of humor when called for 

AA. he motivated me to do my best work 

A5. he discussed his actions, decisions, and 

selection of topics 
A6. he wade difficult topics easier to. understand 
A7. he suncsarized major points 
A8. he stated objectives for each class sessioc 
A9. he discussed practical applications 

50. avoided excessive nathcmatical details 

51. Any other outstanding positive qualities? 
Mark Mock (a) if yes, and specify on the 
reverse side of answer sheet 

52. The course presented facts and concepts from related fields - 

(a) most of the time 

(b) some of the time 

(c) very infrequently 

(d) never 

(f) no answer (i.e. leave blank)' 



(a) 


yes 


(a) 


yes 


(a) 


yes 


(a) 


yes 


(a) 


yes 


(a) 


yes 


(a) 


yes 


(a) 


yes 


(a) 


yes 


(a) 


yes 


(a) 


yes 


(a) 


yes 


(a) 


yes 


(a) 


yes 


(a) 


yes 



53. In reference* to question 51, I prefer - 

(a) much more 

(b) more 

(c) about right 

(d) less 

(e) much less 

(f) no answer 

5A. The lecturer related class topics to students' lives and concerns, and 
more generally to the outside world - 



(a) as often as possible 

(b) a lot of the time 

(c) some of the time 

(d) very infrequently 

(e) never 

(f) no answer 

55. In reference to question 53, I prefer - 

(a) much more 

(b) more 

(c) about "^ight 

(d) less 

(e) much less 

(f) no answfcr 

56. Due to my taking this course, my mastery of the subject matter and of the 
relevant skills and methods 



(a) have improved greatly 

(b) have improved somewhat 

(c) have remained constant 

(d} have decreased somewhat due to confusion created by the courS|^ 

(e) have decreased a great deal 

(f) no answer 



57. Do you think this Questionnaire asked the right questions? If yes mark (a) 
It not, nark (b) and specify on the reverse side of answer sheet, - 





(b) no (specify on the reverse side of answer sheet,) 
I Drefer not ro aneuo'r 



58. 



(») yes 

(b) no ( ^ 

(^) I prefer not to answer 

iJ^f^^Kifr"?! °" ^^"^ ^''''^ °^ ^^"^ ^'^^^ anything you particularly 

^JilenS """^ particularly disliked. - Did you make such a 



(•) yes 

(f) no (i.e. leave blank) 
59. Your class is - 

(a) fresh-an (b) sophomore (c) junior 

(d) senior (e) graduate or special student (f) i prefer not to answer 

W» Your major is in the area of - 

(a) engineering 

(b) premed or life sciences 

(c) nathematics 

(d) chemistry 

m ?*n~fil!*'^,''r"' °" °f th« sheet) 

ItJ 1 prefer not to answer ^ — - 

61. fkm B»ny hours of work outside class did you put in on this course' 
(exclude hours related specifically -o lab). - course. 

(a) between 0 and 2 hours per week 

(b) between 2 and 4 hours per week 

(c) between 4 and 6 hours per week 

(d) between 6 and 8 hours per week 

(e) over 8 hours per week 

(f) I prefer not to answer 

62» In the course do you expect to get - 

(a) a good grade 

(b) an average grade 

(c) a poor grade 

(f) I prefer not to answer (i.e. leave blank) 

63. How many classes of this instructor have you missed - 

W none (b) veiy few (c) few 

f*^) "•^"iy all (f) I prefer not to answer 

^* You arc taking this course - 

(a) to fulfill a General Education requirement 

(b) to fulfill a requirement of my program 

(c) elective, chosen because of my interest in the subject 

(d) elective, chosen because I needed an easy course 

(e) elective, chosen because I heard the course or professor was good 

(f) 1 prefer not to answer ^ 



Of the courses that would have served my purpose, this 

(a) only choice that fulfilled my needs 

(b) my first choice 

(c) my second choice 

(d) only course still open 

(f) no answer (i.e. leave blank) 



UNIVERSITY OF MARYLAND 

Physics Departmenc 
Teaching Questionnaire 

Form 3 - "Physics" (or G.P.S.) Graduate or Undergraduate Courses (Lecturer) 



The purpose of this questionnaire is to obtain in an anonymous fashion your 
opinion about the physics and astronomy teachers and courses. This infor- 
mation will be used in many ways — to improve each teacher's performance, 
to help the chairman assign faculty members to courses, and to provide 
important input in the evaluation of faculty with respect to retention and 
promotion. 

Do noK put your student numbe r in the box labelled student number in the lower 
right hand corner of your answer sheet Instead put the code number for this 
courst: (which should be on the blackboard) in that nine digit space by block- 
ing in the appropriate blocks, e.g. Code Number 012210021 would become: 
Your instructor will get the tabulated results of this ' j^jj^^^i^,^^ 

questionnaire, together with any written comments, numbj f< 

after the final grades in this course have been ' ^ " ' ' 

submitted. 0 f Xl | CT^X ( 



General Instructions 



■ - - / / - \ * - • 

0 * C 0 o ■ ■!> o O 
"2.1 X I ■ X -x % t 



All entries must be on the STANDARD ANSWER SHEET. 
Comments must be written on the back of this sheet. 

Use a Number 2 pencil only . Each question must have 
either one block only or no block marked. Multiple 
answers are not allowed, so that a wrong answer 
must be erased carefully and completely. To mark 
the response (f) leave all spaces blank. This will 
be counted. 



: I 
I I 



3. Black in your course code number in the nine digit space labelled"Student 
Number" on your Standard Answer Sheet. 

A. Finally, please write on the top of the Standard Answer Sheet the c<^urse 

number; section number; name of the specific person whom you are evaluating 
and whether this is a lab, recitation, or lecture; (e g. , "Phys 100, 
Section 201, John Doe,- Lab.") This information shouLi also be 'on the 
blackboard. 



Even if you do not fill out the questionnaire please carry out instructions 3 
and A , and return the otherwise blank answer sheet. Thank you for your 
cooperation. 



A1.14 

Qttettlons: 

!• With regard to this questloimaljre: 

(a) you basically approve of such questionnaires, and hence you are 
responding to the vhble questionnaire, 

(b) You find such questionnaires an Imposition, but In spite of this 
. you are resnondlng to the whole questionnaire. 

(f) (l.e# leave blank) You disapprove of such questionnaires, and 

hence (PLEASE, after filling In the code number) Intend to abstain 
conpletely. 

2. 1 evaluate U.c lecturer's k„owlod«e of the subject as being - 
(a) exec Client • fu^ 

(d) less than satisfactory f''*^ satisfactory 

If) no answer ^ '""^ incompetent 

3- His ability to get the material across to mc is - 
(2) excellent fK^ 

(d) poor }:\ satisfactory 

(f) no answer 

His prcparatlCM for class Is - 

P«>' (b) satisfactory (c) good 

(d) less than satisfactory (e) excellent (f ) no answer 



5. TTie lecturer - 

(a) usually started and ended his class on time 

(b) sometimes started late or ended late 

(c) frequently started late or ended late 

(d) frequently started late and nlso ran over 

(e) oft I calls off class or doesn't show up 

(f) no answer 

6. TTie course as a whole was planned and organized - 

(a) extremely well (b) well • fc) acceptably 

(d) rather poorly (e) extremely poorly (f) no answer 

7. In my opinion the inclusion in the course of lecture demonstrations, 
film»> and visual aids was - 

(a) too much (b) good (c) adequate 

(d) not enough (e) too little (f) no answer 

S. The lecturer - 

(a) seldom made the underlying pliysical ideas clear 

(b) nevcx* made the underlying physical ideas clear 

(c) usually made the underlying physical ideas clear 

(d) always made the underlying physical ideas clear 
(f) no answer (i.e. leave blank) 

9. TTie lecturer's ability to make the subject interesting - 

(«) excellent (b) good (c) satisfactory (d) poor 

(e) zero (f) no answer 0^ 

10- The lecturer's apparent concern in teaching this class showed that he was - 

(a) exceedingly interested 

(b) usually quite interested 

(c) fairly interested 

(d) only occasionally interested* 

(e) bored 

(f) no answer 

His attitude toward questions in class was - 



(a) very encouraging (b) usually encouraging (c) adequately receptive 
(d) flippant (e) discouraging (f) j have no opinion 



From my experience, the teacher's availability for questions outside 
of class hours was - 



(a) encourapi nj», and j:enerous of his time 

(b) u*:unlly available 

(c) barely adequate 

(d) discourai:i»j', anJ ii<;ual!y unavailable 

(e) always una\aiin!)lc 

(f) 1 have no opinion 

The feeling between the lecturer and the students was - 

(a) that of strong good will (b) that of good will 

(c) neutral (d) negative 

(e) antagonistic (f) no answer 



Does the lecturer raKe the distinction between the important points of the 
subject and incider,tal infor.'^.ation - 

(a) yes, he always nakos that distinction very clearly 

(b) he r?a'c:: tliat distii.ction clear in most instances 

(c) somctint'S he doc*:, so;retir.os lie doesn't 

(d) for the Rost part such a distinction is not made 

(e) never 

(f) no answer 

The exams - 

(a) always covered material or techniques emphasized in the course 

(b) usually covered natcrial or techniques emphi^ized in the course 

(c) sor.etimes covcreJ r.aterial or lecliniques emphasized in the course 

(d) frequently covered natcrial not em;*hasized in the course 

(e) were totally unrelated to the course 

(f) no answer 

The exams and gradinj; ii: the courf.e were - 

(a) much too easy; easy to ^et p.ood f^rade without adequate understanding 

(b) a little too easy 

(c) very fai r 

(d) sonicihat loo diffrri'U to yst a {^ood grade 

(e) much too hard 

(f) no ar-.wcr 

The ass j;,nme:»is in the coi rse uere - 

(a) muc!i loo hard or much too Uav^ 

(b) I'eiicrally somevvh.»t to'^ dirficull or somewhat too long 

(c) about rij:ht foi the course 

(d) not quite ;;de(piaie 

(c) non-existent or grossly inadequate 

(f) no answer 

The problems and/or questions assigned were « 

(a) very lielpful in under :.:nding the material 

(b) helpful jn unccrstn»uliii|- the r<'aterial 

(c) soi:,euhat helpful in unilerstandi »ig the material 

(d) generally not well selected 

(e) poorly selected 

(f) no answer 

The lext(s), hando'Jls, rcadii.r.r, etc, were - 

(a) excellent (b) good (c) satisfactory 
(d) less than satisfactoiy (e) poor (f) no answer 

TJic grading and co'.Mcntary on the horiowork was - 

(n) carcn*!ly doj^e and heJpful 

(b) adc(juately (^o)k» 

(c) less than satisfactory 

(d) arbitrary and vifair 
(o) not done 

(f) no answer 



If yon were to take another pl'vsics course nt approxir.atcly the sanc^ 
level in which this person were teaching, you would - 

(a) eagerly seel: his section out 

(b) lake his section if convenient 

(c) have no stronjj fcelinps 

(d) avoid his section if easily possible 

(e) avoid liim like the plague 

(f) no ansver 



22. On the basis of this person»s teaching in this course I think that he - 

(a) nerits appronriate recognition for his truly outstanding teachinn 

(bj IS a good teacher 

(c) has the potential of becoming a good teaclier 

(d) is an adequate teacher, but without rruch potential 
(c) should not be retained as a teacher in this course 
(f^ T»r» 'iim\,vr 

23. The math and/r-- physics background implied by the prerequisite of thio 
course was - 

(a) sufficient to handle the course 

(b) inadequate, but the instructor filled in the gaps 

(c) inadequate am[ further, the instructor failed to fill in the gaps 

(d) greater than needed 

(e) poorly related to the cci.rse 

(f) I don't know tlie official prcreciuisite 

24. In my opinion the general level nwd speed of the course verc - 

(a) much too hard and/or much too fast 

(b) soii;e\:hat too hard and/or soMcwhat too fast 

(c) just right 

(d) a little too easy and/or a little too slow 

(e) much too easy and trivial 

(f) no answer 

25. His use of examples and illustrations t:as - 

(a) excellent and eifective 

•(b) good 

(c) adrqo.it e and lielpful 

(d) less than sjitit factory 

(e) notice.ibly Inclcing 
' (f) no a?.si;er 



For each cf the followi.ig possible b;,d ]i;:bit^, p; ii'1 b locV (a) if rnd c r.ly if 
the instructor suffers fro:i; it, and it would be^d sjriiulc ioir fun to cl-Vrect 
it; otlieivise make no mark - 



26, inaudible ' (a) 

27, writing vj:s illegible (a) 

28, spoke cither too Tar.t ov too slow (a) 

29, monotonous style of r-pc.jkinj: (a) 

30, poor orj'jni cation on blackboard (a) 
31 • lectured to the blackboard (a) 
52, paced too nucli (yj 

33. "hums" and Vhaws'* (a) 

34. distracting or nervous mannerisms (a) 

35. caustic or sarcastic nr-nnjr (a) 

36. any other bad habits? Mark (a) if 
"yes", and specify on tlie reverse (a) yes 
side of answer sheet 



should be corrected 
should 1)0 corrected 
should be corrected 
should be corrected 
should be corrected 
should be corrected 
should he corrected 
should be corrected 
should be corrected 
should be corrected 



h'" iK/dJSo"^- ^''^ --"-tor 

37. is articulate , n 

38. hns c.\cc)3cn: or^'.oniz.a ion on blncl.bo.ird fal 

39. has very kcU pnccd )rcturo<; r^i ^ l 

40. has a lively .style of .pe.ikinn Hi 

41. b:is chc.rlul or plo..-..m Jispositio.i , 

42. speaKr c]>,oc:l> to ihv c].•;^ 

43. h;iG a sen^c of humor uh.-n called for v^c 

44. be motiv.iiC'J me to do iv host \.orK (A l 

K** J yes 

45. he discussed his actions, decisions, and 

selection of topics (a) yes 

46. he made difficult topics easier to understand (a) yes 

47. he summarized major points (g) yes 

48. he stated objectives for each class session (a) yes 

49. he discussed practical applications (a) yes 

50. avoided excessive mathematical details (a) yes 

51. Any other outstanding positive qualities? 

Mark block (a) if yes, and specify on the (a) yes 

reverse side answer c^'^et 

52 • Do you tmnk this Questionnaire asked the right questions? If "yes", nark 
(b)» If not, mark (b) and specify on the reverse' side of ansver sheet* - 

fa) yes 

lb) no (mark (b) and speciry on the reverse side of answer sheet)* 
(f) I prefer not to answer, ' 

53» Please discuss on the back of the answer sheet anything you particularly 
liked about the course or particularly disliked* Did -------- 

See 1.^ 

54. Your class is - 

(a) freshman or first year graduate 

(b) sophomore ©£ second year graduate 

(c) junior o£ third or higher year graduate student 

(d) senior 

(e) special student 

(f) I prefer not to answer 

55. Your major is in the area of - 

(a) physics or astronomy 

(b) engineering 

(c) life sciences or premeJ 

(d) mathematics 

(e) other (mark here, and specify on the reverse side of the sheet) 

(f) I prefer not to answer 

56. How many hours of work outside class did you put in on this course? 
( exclude hours related specifically to lab) - 

(a) between 0 and 2 hours per week 

(b) between 2 and 4 hours per week 

(c) between 4 and 6 hours per week 

(d) between 6 and 8 hours per week 
le) over 8 hours per week 

(f) I prefer not to answer 

57. In the course do you expect to get - 

(a) a good grade 

(b) an average grade 

(c) a poor grade 

(f) I prefer not answer (i.e. leave blank) 

58. How many classes of this instructor have you missed - 



(a) none (b) very few (c) few 

(e) nearly all (f) I prefer not to answer 



(d) many 



A1.18 

3.6 

59. You are taking this course - 

(a) to fulfill a General Education requirement 

(b) to^fulfill a requirement of my program 

(c) elective, chosen because of my interest in the subject 

(d) elective, chosen because I needed an easy course 

(e) elective, chosen because I heard the course or professor was good 

(f) I prefer not to answer 



60. 



Of the courses that would have served my purpose,' this one was - 

(a) only choice that fulfilled my needs 

(b) my first choice 

(c) my second choice 

(d) only course still open 

(f) no answer (i.e. leave blank) 



I 
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UNIVERSITY OF MARYLAND 

Physics Department 
Teaching Questionnaire 



Form 4 - Recitation Seer ions 
All ,u..Lions i.tor to the recitation, and to the recitation T/A or instructor. 

The purpose of this qm-st i onn.ii re is to obtnir. in an anonymous fashion your 
oi.iiijoti jLouL lUc piiysics anci astronomy ttdchcrs and courses. This infor- 
matiuu will he used in many ways — to improve each teacher's performance 
to h.-Jj. the duiirman assign faculty members to courses, and to provide * 
important input in the evaluation of faculty with respect to retention and 

prOilDt j DIU 

5fJ^2Lii:i^'.X^iir.«i2i'ieriL_mi^^^ in the box labelled student number in the lower 
right hand corner of your answer sheet. Instead put the code number for this 
course (which should h« on the blackboard) in that nine digit space by block- 
ing xn the approprint- blocks, e.g. Code Number 012210021 would become- 
Your Instructor wi U .-.cl the tabulated results of this 
questionnaire, together with any written comments, "mJlmT 
after tlie final ;;rad_es in this course ha ve been "~- ; -y— r-T-n" r-"< 

submitted. - - . 0 \ I OX)i ( 

"T r rn -vri 



General Instructions 
1 



All entries must be on the STANDARD ANSWER SHEET. 
Comments must be written on the back of this sheet. 

Use a Number 2 pencil only . Each question must have 
either one block only or no block marked. Multiple 
answers are not allowed, so that a wrong answer 
must be era.sed carefully and completely. To mark 
the response (f) leave all spaces blank. This will 
be counted. 



3. Black in yout course code number in the nine digit space labelled"Student 
Number on your Standard Answer Sheet. 

4. Finally, please write on the top of the Standard Answer Sheet the course 
number; section iiimiber; name of the specific person whom you are evaluating; 
and whether this is a lab, recitation, or lecture; (e.g., ^'Phys 100 



Section 20J 
blackboard. 



John Doe, Lab.") This information should also be on the 



Even if you do _>iot_ fi ll out the questionnaire p lease carry ou t instructions 3 

andJi^, and return the otherwise blank answer sheet. Thank you for your 

cooperation. . ^ 
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k.2 

ftie«tion«; 
!• Vlth reeard to this questionnaire: 

(a) You baaicaUy ai>prove of auch quettionnairee, and hence you are responding 
\o the whole questionnaire, pv«u*ng 

(b) You find such quest ionnairea an impoaition, but in apite of thia view 
you are reapondlng to the whole questionnaire. ' 

Jwwci**''!^^''!??,/'^ disapprove of such questionnaires, and hence 
(PUEASE, after filling m the code maber) intend to abstain completely. 

2. I cvalm.tc the lecturer's kno-.^lcilj'c of the subject cs bcinj - 

(aj excellent (b) ^\oo^ (c) sat i:; factory 

(d) less than sat is factor)* (c) incorpctcnt 
(£) no ansuvr 

3. His ability to get the material across to me is • 



(a) excellent 
(d) poor 



(b) fiood 
(e) zero 



(c) satisfactory 
(fj no answer 



k. Xia preparation for class is - 

(a) Poor (b) aatiafaetory (c) good 

(d) leas than satiafsctory (c) excellent (f) no answer 



5. The teacher - 

(a) usually strntcd and ended his class on time 

(b) sometiKcc started late or ended late 

(c) frequcPtJy started late or ended late 

(d) ' frequently started late and also ran over 
(c) often calls off class or doesn't show up 
(f) no answer 

6. The teacher related the recitation class to the lectures, assignments and exams - 

(a) extremely well (b) better than average 

(c) average (d) rather poorly 

(f) extremely poorly (f) no answer 

7. Ihe teacher*s ability to rAike the subject interesting was - 

(a) excellent (b) good (c) satisfactory (d) poor 

(e) zero (f) no answer 

t. The teacher's apparent cowccm in teaching this class showed that he was 

(a) usually quite interested 

(b) only occasionally interested 

(c) fairly interested 

(d) bored 

(e) exceedingly interested 

(f) no answer 

9. His attitude toward questions in class was - 

(a) very encouraging (b) usually encouraging 

(c) adequately receptive (d) flippant 

(e) discouraging (f) i have no opinion 

10. From my experience, the teacher's availability for questions outside of 
class hours was - 

(a) encouraging and generous of his time 

(b) usually available 

(c) ' barely adequate 

* (d) discouraging and usually unavailable 

(e) always unavailable 

(f) I have no opinion 



Al,21 



14. 



II. The feeling between the teacher and the students was 



(a) that of strong good will 

(c) neutral 

(e) antagonistic 



(b) that of food will 
(d) negative 
(f) fto answer 



12. In discussing homework or exa« problems the teacher • 

(a) always makes the underlying physical ideas clear 

(b) usually makes the underlying physical ideas clear 

(c) seldom makes the underlying physical ideas clear 

(d) never makes the underlying physical ideas clear 
(f) no answer (i.e. leave blank) 



IS. "Hie teacher's 



(«) 

(b) 
(c) 
(d) 
(e) 
(f) 



attempts to answer questions about the lecture was 



very clear and to the point 
often clear and helpful 
usually satisfactory 
less than satisfactory 
generally unsuccessful 
no answer 



Class participation . 



(a) 

(b) 
(c) 
(d) 
(e) 
(f) 



IS. 



effectively involved everyone 
involved many people 
involved only a few people 
rarely involved anyone 
was totally lacking 
no answer 

J2hi„g:7oI°wou?S ^" -^'^ this person 



(•) 

(b) 
(c) 
(d) 
(e) 
(f) 



WRcrly seek his section out 

take his section if convenient 

have no strong feelings 

avoid his section if easily possible 

avoid him like the plague 

no answer 



16; 



On the basis of this person's teachine {„ 

P rson s teacning in this course. 1 think that he - 



(•) 

(b) 
(c) 
(d) 
(e) 
(f) 



is an outstanding teacher 
is a good teacher 

has the potential of becoming a good teacher 
no answer 



17. His use of examples ,nd illustrations 



was 



(a) 

(b) 
(c) 
(d) 
(e) 
(f) 



excellent and effective 
good 

adequate and heJpful 
less than satisfactory 
noticeably lacking 
no answer 



For each of the following possible bad habits, urk block fal if -nH o„i„ 



IS. 

19. 
20. 
21. 
22. 
23. 
24. 
25. 
26. 



inaudible 

writing is illegible 
speaks too fast 
speaks too slow 

poor organization on blackhoan: 

talks to the blackboard 

••hums" and 'liaws" 

distracting or nervous mannerisms 

any other bad habits? Mark (a) if 

"ycs«\ and specify on the reverse 

side of answc-r sheet 



(a) 
(•) 
(a) 
(a) 
(a) 
(a) 
(«) 
(a) 



should be 
should be 
should be 
should bt 
should be 
should be 
should be 
should be 



•torrected 
corrected 
corrected 
corrected 
corrected 
corrected 
corrected 
corrected 



(«) yes 



For each of the following positive qualities, mark block (a) if the 
instructor has it to a high degree - 

27. is articulate (a) yes 

2S» has excellent organization of the blackboard (a) yes 

29. has a cheerful or pleasant disposition (a) yes 

30. speaks directly to the class (a) yes 



31. has a sense of sense of humor when called for (a) yes 

32. is very honest when he has made a mistake (a) yes 
or does not know the answer 

33. avoids excessive mathematical detail (a) yes 

34. Any other outstanding positive qualities? 

Hark block (a) if yes, and specify on (a) yes 

the reverse side of answer sheet 



35# Ito you think this Questionnaire asked the rl^t questions? If yes, 

■axk (a). If not, mark (b) and specify on the reverse of anover sheet. - 

|a) yes 

bJ no (mark (b) and specify on the reverse aide of answer sheet). 
I f) I prefer not to answer* 

36* Please discuss on the back of the answer sheet aqythlng you particularly 
liked about the couzje or particularly disliked* See 1*$8 



37. Your class is - 

(a) freshman 

(b) sophomore 

(c) junior 

(d) senior 

(e) griduate or special student 

(f) I prefer not to answer 

38^ Your major is in the area of - 

(a) engineering 

(b) premed or life sciences 

(c) mathematics 

(d) dicmistry , ^ ^ * 

Cc) other fmark here, and specify on the. reverse side of the sheet} 

1 prefer not to ciiswcr 



39» In the course do you expect to get 

(a) a good grade (b) an average grade 

(c) a poor grade (d) I prefer not to answer 

(i.e. leave blank) 



40. How many classes of this instructor have you missed - 

(a) none (b) vexy few (c) few 

(d) many (c) nearly all (f) I prefer not to answer 

41. You are taking this course - 

(s) to fulfill a General Education requirement 

0>) to fulfill a requirement of my program 

(c) elective, chosen because of my interest in the subject 

(d) elective, chosen because 1 needed an easy course 

W elective, chosen because I heard the course or professor was £Ood 

(f) no raply * 

42. Of the courses that would have served my purpose, this one was - 

(a) only choice that fulfilled my needs 

(b) my first choice 

(c) my second choice 

(d) only course still open 

(f) no reply (i.e. leave blank) 
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UNIVERSITY OF MARYLAND 

Physics Department 
Teaching Questionnaire 

Form 5 - "Structured" Laboratory Sections or Courses 
All Questions Refer to the Laboratory 

The purpose of this questionnaire is to obtain in an anonymous fashion your 
opinion about the physics and astronony teachers and courses. -This infor- 
mation will be used in many ways — to improve each teacher's performance, 
to help the chairman assign faculty members to courses, and to provide 
important input in the evaluation of faculty with respect to retention and 
promotion. 

Do not put your student number in the box labelled student number in the lower 
right hand comer of your answer sheet. Instead put the code number for this 
course (which should be on the blackboard) in that nine digit space by block- 
ing in the appropriate blocks, e.g. Code Number 01221002lwould become: 
Your instructor will get the .tabulated results of this 
questionnaire, together with any written comments, 
after the final grades in this course have been 
submitted. 



NUMBER 



General Instructions 
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II 



1. All entries must be on the STANDARD ANSWER SHEET. 
Comments must be written on the back of this sheet. 

2. Use a Number 2 pencil onl> . Each question must have 
either one block only or no block marked. Multiple 
answers are not allowed, so that a wrong answer I 
must be erased carefully and completely. To mark 

the response (f) leave all spaces blank. Thi* will 

be counted. 

3. Black in your course code number in the nine digit space labelled"Student 
Number" on your Standard Answer Sheet. 

4. Finally, please write, on the top of the Standard Answer Sheet the course 
number; section number; name of the specific person whom you are evaluating; 
and whether this is a lab, recitation, or lecture; (e.g., •'Phys 100, 
Section 201, John Doe, Lab.") This information should also be on the 
blackboard. 



Even if you do not fill out the questionnaire please carry out instructions 3 

4 , and return the otherwise blank answer sheet. Thank you for your 
cooperation. 
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Questions t 

!• With reseurd to thie questionnaire: 

(a) Yofu basically approve of such questionnaires, and hence you are 
responding to the whole questionnaire, 

(b) You find such questionnaires an Imposition, but in spite of this 
view, you are responding to the whole questionnaire, 

(f) (i#e» leave blank) You disapprove of such questionnaires, and hence 

(PLEASE, after filling in the code number) intend to abstain completely* 

2, l!he instructor's familiarity with the equipment was • 

a) excellent (b ) good (c ) adequate 

d) weak (e) noticcca>ly lacking (f) no answer 

3* The Instructor's knowledge of the physics of the experiment was • 

a) excellent (b ) good f c) adequate 

d) weak (e) noticeably lacking (f) no answer 



4. In the laboratory the instructor - 

(a) refuses to help students 

(b) tends to spend his time with a few people 

(c) was adequately helpful 

(d) was present, but largely i^orcd the laboratory 

(e) circulates and is very helpful 

(f) frequently leaves the laboratory 

5. Grading and coninents on laboratory reports were - 

(a) carefully done and helpful 

(b) adequately done 

(c) less than satisfactory 

(d) arbitrary and unfair 

(e) not done 

(f) no answer 

6. The instructor's apparent concern in teaching this class shows that 
he was - 

(a) exceedingly interested 

(b) usually quite interested 

(c) fairly interested 

(d) only occasionally interested 

(e) bored 

(f) no answer 

7. The feeling between the instructor and the students was - 

(a) that of strong good will (b) that of good will 
(c) neutral (d) negative 

(e) • antagonistic (f) no answer 

S. The experiments were - 

(a) adequately coordinated with the lecture 

(b) unrelated to the lecture 

(c) poorly coordinated with the lecture 

(d) no lecture - so inapplicable 

(e) well coordinated with the lecture 

(f) no answer 

9. The experiments were - 

(a) interesting and informative 

(b) fairly interesting but not well developed 

(c) dull 

(d) much too detailed so that one lost the forest for the trees 

(e) some good» some bad 

(f) no answer 
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10. The laboratory manual or handouts were - 

(a) very clear and helpful 
0>) generally good 
(c) adequate 
(u) less than adequate 
(e) generally useless 
no answer 



n. 



Jea^ln^ryoS^^SS this person was 



eagerly seek his section out 

tftkc his section if convenient 

have no strong feelings 

a -ia his section if easily possible 

avuid hiia like the plague 



12. Dd you think this Questionnaire asked the right questions? If "yes", mark (a). 
If not, nark (b) and specify on the reverse side of answer sheet. 

(a) yes 

(b) no (nark (b) and specify on the reverse side of answer sheet), 
(f ) I prefer not to answer. 

13 • Please discuss on the back of the answer sheet anything you partlctxlarly 
liked about the course or particularly disliked. See I.58 



14. Your class is - 



(a) freshman or first year graduate 

(b) sophomore or second year graduate 

(c) junior or third or higher year graduate student 

(d) senior 

(e) special student 

(f) I prefer not to answer 

15. Your major is in the area of - 

(a) physics or astronomy 

(b) engiii'.ering 

(c) life sciences or premed 

(d) mathematics 

(e) other (mark here, and specify on the reverse side of the sheet) 

(f) I prefer not to answer 

16. How many hours of work outside the lab period did you put in on work 
specifically related to lah ? - 

(a) none 

(b) between 0 and 1 hour per week 

(c) between 1 and 3 hours per week 

(d) between 3 and 6 hours per week 

(e) more than 6 hours per week 

(f) I prefer not to answer 

17. In the course do you expect to get - 

(a) a good grade (b) an average grade 

(c) a poor grade (f) j prefer not to answer (i.e. leave blank) 

18. How many classes of this instructor have you missed - 

(a) none (b) very few (c) few 

(d) many (c) nearly all (f) I prefer not to answer 

19. You are taking this course - 

(a) to fulfill a General Education requirement 

(b) to fulfill a requirement of my program 

(c) elective, chosen because of my interest in the subject 

(d) elective, chosen because I needed an easy course 

(e) elective, chosen because I heard the course or professor was eood 

(f) no reply ^ 
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20. 



5.1» 



Of the courses that would have served «y purpose, this one was 



(•) 

(b) 
(c) 
(d) 
(f) 



only dioice that fulfilled my needs 

my first choice 

■y second choice 

only course still open 

no rfply (i.e. leave blank) 



ERIC 
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UNIVERSITY OF MARYLAND 

Physics Departnent 
Teaching Questionnaire 
Form 6 - "Unstructured Laboratory Courses" 
All Questions Refer to the Laboratory 

The purpose of this questionnaire Is to obtain In an anonymous fashion your 
opinion about the physics and astronomy teachers and courses. This infor- 
mation will be used In many ways - to Improve each teacher's performance, 
to help the chairman assign faculty members to courses, and to provide 
Important Input In the evaluation of faculty with respect to retention and 
promotion* 

^"""^ labelled student number In the lower 

right hand comer of your answer sheet. Instead put the code number for this 
course (which should be on the blackboard) In that nine digit space by block- 
ing In the appropriate blocks, e.g. Code Number 012210021 would become- 
Your Instructor will get the tabulated results of this L:...^^ 
questionnaire, together with any written comments, 
after the final grades In this cours e have been ~T"-r-i — 
submitted . Q f ,'\,X 



General Ins true tlons 



1. 



2. 



All entries must be on the STANDARD ANSWER SHEET. 
Comments must be wrlt,ten on the back of this sheet. 

Use a Number 2 pencil only . Each question must have 
either one block only or no block marked. Multiple 
answers are not allowed, so that a wrong answer I 
must be erased carefully and completely. To mark 
the response (f) leave all spaces blank. This will 
be counted. 



! I 



( 



I 



3. 



Black In your course code number In the nine digit space labelled"Student 
Number" on your Standard Answer Sheet. 

Finally, please write on the top of the Standard Answer Sheet the course 
number; section number; name of the specific person whom you are evaluating; 
and whether this Is a lab, recitation, or lecture; (e.g., "Phys 100, 
Section 201, John Doe, Lab.") This Information should also be on the 
blackboard. 



Even If you do not fill out the q u estionnaire please carry out Instr uctions 3 

and_4, and return the otherwise blank answer sheet. Thank you for your 

cooperation. ' 



II 



if 
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Qttcttiona : 

1. With regard to this questionnaire: 



(•) Tou btalcilly approve of such queatlonnalrea, and hence you are 
responding to the whole questionnaire. 



You find such questionnaires an iaposition, but in apite of this 



(b) . ^ 

V'^' "«P0"'1^"8 to the whole questionnaire, 

(f) (I.e. leave blank) You disapprove of such questionnaires, and 

hence (PLEASE, after filling In the code number) Intend to abstain 
completely. 



I IMPORTA 
I only th 



ERLC 



^^^^^ "^^^^ eYilumting the laboratory teAching assistant, answti 
onlythe starred questions. 

2. Equipment for experijnents was 

(a) adequate 

(b) inadequate 

(f) no answer (i.e., leave blank) 

3. Equipment for ejqjeriTcnts was 

(a) adequately docunented or e>qplained 

(b) inadequately dociroented or explained 
(f) no answer (i.e., leave blank) 

The instructor's preparation for class was 

?^*u . satisfactory (c) good 

(d) less than satisfactory (e) thorx>ugh (f ) So answer 

*5. The instnictor's kncwledgp of ifhe physics in the e)q)erijnent was 

(a) excellent (b) good (c) adequate 

(d) weak (e) noticeably ladcing (f) no ai^wer 

*6. In the labai^atory the instructor 

(a) was always available and willing to help 

(b) was adequately helpful 

(c) was present, but largely ignored the laboratory 

(d) refused to help students 

(e) frequently left the laboratory 

(f ) no answer 

*7. When I had questions about the equipment, the instructor 

Si *° i^^P ^ ^ *° e^rt knowledge 

(b) shewed less than adequate concern or interest 

(c) Ignored my pleas for help 
(f) no answer (i.e., leave blank) 

*a. The instructor's apparent oonoem in teaching this class showed that he Wd: 

(a) usually quite concerned 

(b) only occasionally interested 

(c) fairly interested 

(d) botr^ 

(e) exceedingly interested 

(f ) no answer 

*9. His ability to get the jMterial acroes to me is 

(a) excellent (b) giod (c) satisfactory 

(d) poor (e) zero (f ) no aiBwer 

*10. .The feeling between the instructor and tiie students was 

(a) that of strong goodwill (b) that of goodwill (c) neutral 
(d) negative (e) antagonistic (f) no answer 
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6.3 

His attitude toward questions in class was 

(a) very encouraging (b) usually encouraging (c) adequately receptive 
(d) flippant (e) discouraging (f ) no answer 

12. Grading and caiments written on laboratory reports wre 

(a) carefully done and helpful 

(b) adequately done 

(c) less than satisfactory 

(d) art)itrary and unfciir 

(e) not done 

(f) no answer 

*13. Ccmnents and criticisms during oral laboratory reports were - 

(a) carefully done and helpful 

(b) adequately dene 

(c) less than satisfactory 

(d) arbitrwy and unfair 

(e) not made 

(f ) no answer 

Ik. Class presentations by the instructor were 

(a) worth the tijne they took 

(b) interesting, but the tijie required could have been better spent 
working in the laboratory 

(c) of little value 
(f ) no answer 

15. The lecturer's ability to make class presentations interesting was 

(a) excellent (b) good (c) satisfactory (d) poor (e) zero 
(f } no answer 

16. The teacher's attenpts to answer questions about the lecture were 

(a) very clear and to the point 

(b) often clear and helpful 

(c) usually satisfactory 

(d) less than satisfactory 

(e) generally unsuccessful 

(f ) no answer 

17. Class participation 



i^l ?f^®ctively involved everyone 

(b) involved many people 

(c) involved only a few people 

(d) rarely involved aiyone 

(e) was totally lacking 

(f) no answer 

t'fSr^°w^Sd ^^'^ course i. vMd, this per.cn was 

(a) eagerly seek his section out — " 

(b) tate his section if convenient 



(c) 
(d) 
(e) 



have no strong feelings 
avoid his section if easily possible 
avoid him lite the plague 



*19. 



On the basis of this person's teaching in this couree, I think that he 

(a) is an outstanding teacher 

(b) IB a good teacher 

(c) has the potential of becoming a good teacher 

W) is an adequate teacher, but without mudi potential 

le} should not he retained as a teacter 

vf) no answer 
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^■a^r^l^ foUowiiig p^i^le had habits, mark block (a) if aivi only 
Sfttc iratructor suffers frcm it, and it would be desirable for hi;n to 
comet It; othervise make no mark 

11: i^tiSIs illegible S should be coi^cled 

22. speaks too fast^ !j f^?"^^ j!! '^"^"^'^ 

2S. W ard ''^J^ ■ g> S T^f.l', 

II: ^r^^tJZi^T ^Tif III ^ 

Vs," and- specify on the reverse 
side of the answer sheet 

[l^L^^ of^the following positive qualities, mark block (a) if the 
anstrvctor has it to a high degree 

29. is articulate (^j 

30. has excellent organization of the (a) ves 

blackboard ^ 

31. has a cheerful or pleasant disposition (a) yes 

32. speaks directly to the class (a) yes 

33. has a sense of humor when called for (a) yes 
3^- 18 very honest when he has mde a mistake 

or does not knew the answer (a) yes 

35. avoids excessive mathematical detail (a) yes 

36. Ary other outstanding positive qualities? 
Mark block (a) if yes, and specify on the 
reverse side of answer sheet (a) yes 

37. lecture presentations by students enrx)lled in the course: 

(a) should be continued 

(b) should not be required 

^""^ SS^^A^*^"^"^ "^^.^ different fomat [mark block (c) 

and specify on reverse side of answer sheet 
(f; no answer 

38- lecture presentations by students enrolled in the course: 
(a) were worth the tijne they tock 

better spent working in the laboratory 

(c) were of little vaue 
(f ) no answer 

Ipwwially found that the time I spent in preparing my cm talk 
and.the e)q)erienoe of present!!^ it were my ctji taix 

(a) worthftMle 

(b) of mar^nal value 
(f } no answer 

I found that the effort spent Cn this course was 

(a) excessive for ±he anomt of cr^t given 

M ^^""1 of equivalent credit 

S as that spent on other courses of equivalent credit 

(d) less than that spent on other equivalent courses ^'^''^^''^ "^^^ 

t£ "S^l^^"^^^^'^ ^ «^3ect natter and 

(ai has ijiproved gr^tly 

(b) has ijiproved scmevftiat 

(c) has renained constant 

S) is r^dS ^ """^'^ ^ 

(f ) no answer 



39. 



HI. 
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U2* The oesurse as a Whole was planned and organized 

(a) extranely well (b) better than avex^ 

(c) about as well as the avex^ course (d) ivi^r poorly 

(e) extremely poorly (f ) no answer 

Disregarding the amount of effort involved (as discussed in 
question UO), I feel that the e)q)ericnce gained in this laboratory 

(a) was of great value (d) was of no value 

(b) was worthwhile (f) no answer (i.e., leave blank) 

(c) was of nax^inal value 

V4. How many hours per week in total did you spend on this course? 

(a) U-8 (e) over 20 

(b) 8-12 (f ) I prefer not to answer 

(c) 12-16 

(d) 16-2C 

US. All things considered, I feel that *Aiat is expect of the student 

in this course 

(a) is reasonable and should be left undianged 

(b) is sancwhat excessive, and should be sHjjhtly reduced 

(c) is greatly excessive 

(f) no answer (leave blank) 

U6. What grade do you expect to receive in this course^ 

(a) A 

(b) B 

(c) C 

(d) D 

(e) F 

(f ) I prefer not to answer 
H7. Your class is 

(a) senior 

(b) first year graduate 

(c) second year graduate 

(d) third or hi^r year graduate 

(e) special student 

(f ) I prefer not to answer 

HB. Specialization ; Before I took -ttiis coirse: 

(a) I planned to be an experimentalist if possible 

(b) I planned to be a theorist if possible 

(c) I was undecided 

(f) I prefer not to answer (i.e., leave blai^) 

H9. IXie to ny taking this course, ny attitude towards experijiental woric 
has 

(a) beoone more favorable 

(b) renained undianged 

(c) become less favorable 

(d) I prefer not to answer (i.e. , leave blank) 

SO. Should this course remain a required graduate course? 

(a) yes, but only for experimentalists 

(b) yes, but only for theorists 

(c) yes 

(d) no 
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If «X f!l?' )^ questions? 

(a) yM 

(a) yes 

(f) no (I.e. leave blank) 
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UNIVERSITY OF MARYLAND 

Physics Department 
Teaching Questionnaire 

¥OTm 7 - Laboratory Course 

All Questions Refer to the Laboratory 

The purpose of this questionnaire is to obtain in an anonymous fashion your 
opinion about the physics and astronomy teachers and courses. This inlvt-^ 
mation will be used in many ways — to improve each teacher's performance, 
to help the chairman assign faculty members to courses, and to provide 
important input in the evaluation of faculty with respect to reter.liou a:id 
promotion^ 



Do not put your stude nt number in the box labelled student number in rhe louo.t 
right hand comer of your answer sheet. Instead put the code number tor this 
course (which should be on the blackboard) in that nine digit space by block- 
ing in the appropriate blocks, e.g. Code Number O1221002lwould become- 
Your instructor will get the tabulated results of this 
questionnaire, together with any written comnents, 
after the final grades in this course have b^ en 
submitted. " 



General Instructions 

1. All entries must be on the STANDARD ANSWER SHEET. 



HUMUHR 

■■]■] I rrrvi 1 



2. 



I t 
til* 



7 XI 2 W JL 



Comments must be written on the back of this sheet. 

Use a Number 2 pencil only . Each question must have 
either one block only or no block marked. Multiple 
answers are not allowed, so that a wrong answer ) 
•ust be erased carefully and completely. To mark 
the response (f) leave all spaces blank. This will 
be counted. 



i I 



I 



r 
I 



3. 



4. 



II 



Black in your course code number in the nine digit space labelled"Student 
Nunber" on your Standard Answer Sheet. 

Finally, please write on the top of the Standard Answer Sheet the course 
nunber; section number; name of the specific person whom you are evaluating; 
and whether this is a lab, recitation, or lecture; (e.g., "Phys 100, 
Section 201, John Doe, Lab.") This information should also be on the 
blackboard. 



Even if you do not fill out the questionnaire please carry out instructions 3 
«nd_4, and return the otherwise blank answer sheet. Thank you for your 
co operation. 

3*|Vlhen evaluating the laboratory Teaching Assistant, answer only the starred - 
(questions 



II 



lERJC 
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jjuestlons: 

1# With regard to this questionnaire: 

(a) You basically approve of such questionnaires, and hence yo>i 
epondlng to the whole questionnaire. 

(b) You find such questionnaires an linpo«5itlon, but In npltr of t 
you are responding to the whole questionnaire. 

(f; (i.e. leave blank) You disapprove of such quent jonnnJr'*' f» 
(PLEASE, after flUlng In the code number; iatcr.u lo <tl). .... 

2« Ihe experiments were - 



Interesting and Infoncatlve 

fairly Interesting but not well developed 

dull 

much too detailed so that one lost the forest for the trees 
Bome goody some bad 
no answer 



3« Ttt laboratory manual or handouts were 



very clear and helpful 
generally good 
adequate 

less than adequate 
generally useless 
no answer 



Ihe instructor's preparation for class was 

(a) poor (b) satisfactory (c) good 

(d) less than satisfactory (e) thorough (f ) no answer 

•5* The instructor's kncwledge of the physics in the experijnent was 

(a) excellent (b) good (c) adequate 

(d) weak (e) noticeably lacking (f) no answer 

In "fte laboratory the instructor 

(a) was always available and willing to help 

(b) was adequately helpful 

(c) was present, but largely ignoi>ed the laboratory 
M refused to help students 

i (e) frequently left the laboratory 

(f ) no answer 

When I had questions about the equipment, the instructor 

(a) was able to help me or direct ma to someone with expert knowlef.^^e 

(b) shewed less than adequate concern or interest 

(c) ignored my pleas for help 
(f) no answer (i.e., leave blank) 

•8. Ihe instructor's apparent concern in teaching this class showed that Uc* was 

(a) usually quite concerned 

(b) only occasionally interested 

(c) fairly interest e J 

(d) boz^ 

(e) exceedingly interested 

(f ) no answer 

•9. His ability to get the material across to me is • 

(a) excellent (b) good (c) satisfactory 

(d) poor (e) zero (f ) no answer 

•10. The feeling between the instructor and the students was 

(a) that of strong goodwill (b) that of goodwill (c) neutral 
(d) negative (e) antagonistic (f ) no answer 
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*n. His attitude towazxi questions in class was 

(a) very encounaging (b) usually encouraging (c) ad^n.^t-ou, 

(d) flippant (e) discoursing (f) ^ anSer ^ ""''"^P""' 

«2. Grading and carments written on laboratory reports were 

(a) carefully done and helpful 

ib) adequately done 

(c) less than satisfactory 

id; arbitrary and unfair 

(e) not done 

(f) no answer 

♦13. Oral comments or criticisms during the laboratory period vere 

Carefully done and helpful 
adequately done 
less than satisfactory 
arbitrary and unfair 
not made 
no ansver 

lU. Qass presentations by the instructor wer« 

(a) worth the time they took 

(b) interesting, but the time required could have been better spent 
working in the laboratory 

(c) of little value 
(f ) no answer 

15. The lecturer's ability to make class presentations interesting was 

(a) excellent t ) good (c) satisfactory (d) poor (e) zerxD 
(f ) no answer 

16. The teadier's atteirpts to answer questions about the lecture were 

(a) very clear and to the point 

(b) often clear and helpful 

(c) usually satisfactory 

(d) less than satisfactory 

(e) generally unsuccessful 

(f ) no answer 

17. Class participation 

(a) effectively involved everyone 
• (b) involved many people 

(c) involved only a few people, 

(d) rarely involved anyone 

(e) was totally lacking 

(f ) no answer 

*18. If you were to take another physics course in wtddh this person was 
teaching, you would ^ 

(a) eagerly seek his section out 

(b) take his section if convenient 

(c) have no strong feelings 

(d) avoid his section if easily possible 

(e) avoid him like the plague 

*19. Oi the basis of this person's teaching in this course, I think that he 

(a) is an outstanding teacher 

(b) is a good teacher 

(c) has the potential of becoming a good teacher 

(d) IS an adequate teacher, but without much potential 

(e) should not be retained as a teacher 

(f ) no answer 



Y.l. 



r«rj21.^ suffers from it, and it would be desirabL^ r.. 

correct itj otheivise make no nari< 

20. inaudible . , , 

21. writing IS Illegible /=) .i.^„. , [ ' 

22. speaks too fast ' '' ' ^ •• ' ' 

23. speaks loo s2a. f"*? '"^ - •• 

t^sTote'Sii^^J*^-^-^^ ^ - • ■ . 

27. distracting or nervous .umnori-.T.c [t] k " \' 

28. any other bad habits? tor:. "; if [t] ^J^^^^ !>-. rr.^.-cr,..-i 
"yes," and specify cr: tiie rv^v-ji-sf 

side of the answer sheet 

^^^s^:^^^ '^'^ the 

29. is articulate ^ 

30. has excellent organization of the 

blackboard ^ " 

31. has a cheerful or pleasaiit disposition (a) yec 
32 



speaks directly to the class (a) ypg 

dd. has a sense of humor when c^"'''-^ ' ' ^ 
is very, honest wh^n he has i 
or does not know the answer 



IS very, honest wh^n he has made a mistake ^ 

or does not knew the answer (a) ves 

35. avoids exc^sive jiBthcmatica] detail (a) ves 
3B. Any other outstanding positive .qualities^ 
Mark block (a) if yes, and specify on the 

reverse side of answer sheet (a) yes 

?7. I found that the effort spent on this course was 

(a) excessive for the anount of credit given 
M "^J^^ ^^^^ °" «^"^es of equivalent c-^^dii 

d 'f^ ^ of equi;^t,t crvEdit 

(d) less than that spent on other equivalent courses ^"^^''^''''^ ^^^^ 

tS J^le'SS^sJS^^^^ — 

(a) has iJiproved.gr^tly 

(b) has ijiproved sanewhat 

(c) has remained constant 

!p! ^"^^^^ sofnewhat due to confusiai created try the com^.c' 

(e) has decreased a gr^t deal <-<^>uioc 

(f ) no answer 

39. The course as a whole was planned and organized 

(a) extremely well (b), better than average 

(c) about as well as the average cburse (d) rather poorly 

(e) extremely poorly (f ) no answer 

kO, ttsregardiJTg the amount of effort involved (as discussed in 

question 40), I feel that the experience gained in this laboratory 

(a) was of great value (d) was of no value 

(b) was worthwhile (f) no answer (i.e., leave blank) 

(c) was of marginal value 

kl. Hew many hours per week in total did you spend on this course? 

(a) i*-8 (e) over 20 

(b) 8-12 (f) I prefer not to answer 

(c) 12-16 

(d) 16-20 

k2. All tilings considered, I feel that v^t is expected of the student 
m this course 

' (a) is reasonable and should be left unchanged 

(b) is sanewhat excessive, and should be slightly reduced 

(c) is greatly excessive 
(f) no answer (leave blank) 



^^3. What gi>ade do you expect to receive in this course? 



(a) A 

(b) B 

(c) C 

(d) D 

(e) F 

(f ) I prefer not to answer 



hh. Do you think this Questionnaire has asked the right questions' 

f/^'i T"" " '"^ ^P^^ify the reverse 

side of the answer sheet. 



(a) yes 

(b) no ( 
(f) possibility 



'"""^ °" °f 'he answer sheet. 



Please discuss on the back of the answer sheet anything you particularlv 
liked about the course or particularly disliked. Suggestions fo '^ 
improvenent, change In format, or enphasis. etc. would be appreciated. 
Did you make such a comment? Kp^'i-iaLea. 



(a) yes 

(f) no (i.e. leave blank) 
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A, 2, List of Courses and Corresponding Questionnaires 



List of Courses and Questionnaire Type. 



/pe Classification 
i> Lecture : 

(General or cul- 
ture oriented) 

1. Lecture : 

(Service or- 
iented) 



Physics Course Numbers 
101, 111, 112, 114, 400, 401 



117, 121, 121H, 122, 122H, 161, 161H, 221, 222, 262, 262H 
263, 263H, 420 



3, Lecture : 

181, 182, 271, 283, 284, 404, 405, 406, 410, 411, 412, 

(Physics Tnajor 413, 414, 421, 422, 423, 431, 441, 443, 451, 461, 463, 465, 471. 

and General Phys- 483, 601, 602, 604, 606, 622, 623, and all higher courses 
ics Science major) 

4. Recitation: 



T/A 



117, 121, 121H, 122, 122H, 161, 161H, 262, 262H, 263, 263H 



5. Laboratory section 

of general course 117, 121, 121H, 122, 122H, 161, 161H, 262, 262H, 263, 263H, 
T/A 

6 . "Unstructured" 



Laboratory 
Course 



429, 621 



7. Laboratory 
Course: 



285, 286, 365, 485, 
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APPENDIX A, 3 - MINOR SUGGESTIONS FOR IMPROVEMENTS TO THE FALL 1971 
QUESTIONNAIRES 

[A reference 2.27 refers to question 27 on questionnaire 2] 
A3. 1 "Editorial" Suggestions 

Move bottom instruction on each questionnaire (unnumbered) to top of 
front page, print it in capitals, and surround it by a border. Revise the 
wording? 

Q 1.61> also 2.61, 3.56> 5.16, 6.44, 7,41 : 

In the "preamble" for each question, add "(FOR SUMMER SESSION, FIRST 
DIVIDE YOUR ANSWER BY TWO, then decide on the "appropriate" interval.)" 

Q 2>20, also 3>20 ; 

Change response f to read "there was no homework, or I prefer not to 
answer." (PRINT BOTH) 

Q 2.53 ; 

Change reference to question 52 
Q 2.55 ; 

Change reference to question 54 

Q 3.53, also 4.36, 5>13 ; 
Word as Q 1.58 

Q 4.12 ; 

Add a response "(e) the teacher did not adequately discuss these" 

Questionnaire 5 : 

Change title to "Laboratory section of general courses - T/A" 

Questionnaire 6 ; 

Change title to "Unstructured Self-contained Laboratory course" 

Questionnaire 7 ; 

Change title to "Structured Self-contained Laboratory course" 

Q 7.1 : 

Put a "star" before this question (It does apply to T/A*s also.) 

Questionnaire 6 heading, also Questionnaire 7 ; 

Delete "All questions refer to the laboratory" 

Q 6. 4, also 7>4 : 

Change "thorough" to "excellent" 

Q 6.43 and 6>44 ; 

Interchange these questions, and correct the cross reference. THIS IS 
WRONG NOW. 
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Q 7.18 : 

add "(f) no answer" 

Q 7.37 : 

add "(f) no answer (i.e. leave blank)" 
Q 7.40 and 7.41 : 

Interchange these questions, and correct the cross reference. THIS 
IS WRONG NOW. 

Q 7.44 : 

Change (f) to "(f) I prefer not to answer (i.e. leave blank)" 
A3. 2 Further. Suggestion^ for More Questions 

On questionnaire 5, add a question after question 11 similarly worded 
to Q4.16. ' 

On questionnaires 1, 2, and 3 ask about whether the pace of the course 
was even or uneven, too fast due to efforts to cover the syllabus, etc? 
Speed at end of course? 

On questionnaires 1, 2, and 3 ask about the speed with which graded 
homework and exams were returned ~ this does reflect upon the instructor? 

Consider whether questions 4 and 8 should be unscrambled. Alternatively, 
repeat them later on in unscrambled order as a test of "consistency?" 

Repeat another question somewhere to test for "consistency?*' 
A3. 3 An "Administrative" Suggestion 



It has sometimes occurred that the "wrong" questionnaire was issued, 
or some other mixup occurred. Unforttmately students and instructors do not 
understand the interpretation of the code numbers, nor their importance. 
But PAIS relies very heavily on the code number, followed by corrective 
detective work (of section 3*7, p. 3.12). To make sure one is dealing with 
the correct questionnaire, we offer two easy suggestions: 

i) Have a (two part)question which asks: 

1. If the number of this questionnaire in the range of 1-5, indicate its 
number here: a). 1, b) 2, c) 3, d) 4, e) 5. 

2. If the number of this questionnaire is in the range 6-10, indicate its 
nxuDber here: a) 6, b) 7, c) 8, d) 9, e) 10. 

11) Alternatively, but less satisfactoiiLy, make sure that each 
questionnaire has a different number of questions. Then scanning of 
original answer sheets provides easy clues on questionnaire number (scanning 
DATAREAD printout does not give this information, but see Appendix J. 1.2, 
p. Jl.l) 
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APPENDIX A4 ~ QUESTIONNAIRE TYPES 

Questionnaire 1 - "General" Courses (Lecturer) - Typically large enrolment 

freshmen or sophomore courses meeting the General* Education Requirement. 
(65 questions) 

Questionnaire 2 - "Service" courses (Lecturer) - typically large enrolment 
freshmen or sophomore courses forming a specific requirement for some 
other Department's major. (65 questions) 

Questionnaire 3 - "Physics" Courses (Lecturer) - all lecture courses for 
physics or astronomy majors, undergraduate or graduate (60 questions) 

Questionnaire 4 - Recitations - to evaluate the recitation section (usually 
taught by a T/A) associated with "general" or "service" courses (42 
questions) 

Questionnaire 5 - Structured Laboratory Sections - to evaluate the lab 

section (usually taught by a T/A)associated with "general" or "service" 
courses (20 questions) 

Questionnaire 6 - "unstructured" lab - to evaluate either the professor or 
the T/A in a separate lab course for advanced students taught in a 
deliberately non-cook-book, non-structured, non-guided manner (proto- 
real-world-research) (52 questions) 



Questionnaire 7 - Laooratory Courses - to evaluate either the professor or 
the T/A in a separate lower level laboratory course. (45 questions) 
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ABCDE ABCDE ABCDE ABCDE ABCDE ABCDE ABCDE ABCDE 

3 8 13 ' 18 23 28 33 38 
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ABCDE ABCDE ABCDE ADCDE ABCDE ABCDE ABCDE ABCDE 
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ABCDE ABCDE ABCDE ABCDE ABCDE ABCDE ABCDE ABCOE 
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ABCDE AB.CDE ABCDE A8CDE ABCDE ABCDE ABCDE ABCDE 

42 47 52 ' 57 ^ 62 67 72 77 

ABODE A3CDE ABCDE ABCDE ABCDE ABCDE ABCDE ABCOE 

43 . 48 53 ^ 58^ 63 68 73 78 

ABCDE ABCDE ABCDE ABCDE ABCDE ABCDE ABCDC ABCDE 

44 49 54 ; 59 64 69 74 79 

ABCDE ABCDE ABCDE ABCDE ABCDE ABCDE ABCDE ABCDE 

45 , ' 50 55 ' 60 65 . 70 75 80 



11 

8 
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I 124 129 134 ; : 139 , 144 149 154 159 

^ ABCOE ABCDE ABCDE ABCDE ABCOt ABCDE ABCOE ABCOE 

5 125 , L .!l3O0 I ^ 135; ! .'140 145 150 155 160 

STANDARD ANSWER SHEET* A frihtio im u.s.a. dsii20»a t^w^ k«m«4 omcALscAmnifccoftPoiiATioN^^.ttM*. mm 
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B* 2. Instructions Issued to the Instructor with Packet of Materials 

Provisional 1972 

Concerning the Teaching Assessment Questionnaires 
Guide for Instructors, (Including TA^s) 

1* Introduction^ General Policy 

This envelope contains the Physics Department teaching assessment ques- 
tionnaire for this semester, as well as answer sheets and pencils. 

Please recall that the use of such questionnaires is mandatory, as part 
of Departmental policy under the supervision of the Department's Teaching 
Excellence Committee. If for any reason you do not wish to issue such ques- 
tionnaires, please consult with the chairman of that committee. 

Please issue this quest iorinaire during scheduled class time, at the 
beginning of a class. (Experience suggests that with planning the whole 
operation need take no more than 15-20 minutes.) Choose the actual class 
period with care, to obtain good and fair participation, and if possible, 
during the week of ^ . 

2. Immediate Instructions 

Please check immediately that you have received the correct forms and 
check the information on the outside of the envelope. If any error is found, 
please Inform Dr. Griggs immediately , and return the envelope to him. Do 
not issue forms. 

Please note specifically the following points: 

a) There are 7 different' formats of questionnaire, to cover the three 
different teaching situations .(lecture/lab/recitation) and according to whether 
professor or TA is being assessed. Make sure you have the right ones. 

b) Check course number, section number, name of instructor or TA. 

c) If two professors or two TA's are involved in the same section, the 
forms are not applicable . Please check with Dr. Griggs for the procedure to 
follow. 

d) Please check the 9 digit code number. The first 3 digits should be the 
course number, the S,6,&7th digits should be the section number, and the 8th 
digit should be the questionnaire ntmiber. 

Please note that there have been various revisions to the procedure and 
to the questionnaire. 



3. Instructions When Actually Issuing Questionnaires 

1. Copy onto the chalk board the entire legend on the brown envelope, 
except the encircled numbers* This is a reminder to the students 
to mark their score sheets appropriately* 

2* Please do not try to get returns from absent students* Those loose 
sheets do not get coded properly, so usually cannot be run through 
the machine and used* 

3* Please ask a student to collect the forms, to put them in the origi- 
nal containers, and to return them to Dr* Griggs* office, Z-115; 
for AstronoB^, Mrs* Mizrahi* (This avoids any suspicion that the 
instructor reads the answer sheets before they are processed and 
before grades are issued * ) 

4* Please collect the pencils so that they can be re-used* We also 

need the questionnaires so that the results can be accompanied by 

copies of the questions* These can be returned either by you, 

separately, or with the answer sheets* 

5* Enter on the packet envelope your best estimate of the number of 

* 

students actually participating in the course at this time (i*e* 
number of officially enrolled students minus number of unofficial 
"dropped out" students - F ) 



to be distinguished from those present on the day the questionnaire is issued* 



TEACHING assi;sshi:nt QUi:sTio:njAiiui:s 

(Guide for Instructors, includinj; TA\s) 



Revised July 3 971 ? 

I. WAIU^TKH ; Do not give out two different questionnaires to each student .a on^* 
time. Chaos \:ill ensuel Tf you us,e t\;o different questionnaires on the r.jine 
occasion (avoid this if possible) make sure one set is coinpletc]y in and the 
code number changed on the blackboard before you issue the second set. 



!!• Classroom Procedure for Instructor 



a) Announce that the questionnaire is to be distributed, ask that students 
who wish to abstain first record their abstention on the form^ (iliey may 
then leave.) 

b) Ask that no one write anything un'-.il instructions have been read out. Dis- 
tribute pencils, qxicstionnaires, answer sheets, and comment sheets. 

c) Prepare on the blackboard a listing of: Course Number, Section Number, name 
of pers<m being evaluated, whother lecture, lab or recitation is being evalu- 
ated, and v:hether the instructor or the teaching assistant is being evalxiated: 
e.g.: 



Phys 100, Section 201, John Doe, Lab, T/A } 



d) Put the code number and "sketch" of blacked in boxes on the board, as sho\vm 
in the sample. USE THE CORRl-CT CODE > as provided on the outside of the envelope. 



e) Ask the students to copy the information in block I onto the top of the 
STANDARD ANSWER SHEET, and to enter the code number as shown in II in the space 
marked ''Student Number*' on the Standard Answer Sheets 

Ask students to read the Revised instructions, and to fill out the standard 
answer sheets. Tell them that if they wish to amplify their responses, or to 
comment upon other items, such comments should be made on the separate "comment 
sheet In that case, they should fill in thd identifying information on the 
comment sheet. 



III. At End 

Collect all answer sheets, whether blank or otherwise, also all comment sheets, 
also pencils. Do not collect questionnaires. 

Put all these materials in the original envelope, seal it > and have a student 
take this envelope to Dr. Griggs — Room Z-115. The use of a student is = 
important » 
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iM'PLNDIX C. Mi\TERI.\LS NEEDED FOR PREPARATION OF AGGREGATES USING AGGFORM 



£^ Heirarchic al Aggregate Tree, also the Header Card Dcrk 

(The "code" is explained at the bottom of p. CI. 3.) 

This tree is dravm directly upon the header card deck, and shows some 
comments, as well as the enrolment in the righcmost columns. 



•iiioioin 

f •11122011J 
I »11401011J 
•400010111 



i 



r 



♦117010121 
•121210123 
•12131012: 
•12:>01O123 
•16'».01O12J 
•221010121 
•26201012? 
•262040121 
•26^060121 
•?63010121 
•42 )01O121 
•420020123 
•42'103O121 
•420040121 



•181010131 
•181020133 
•181030131 
•181040131 
•181050133 
•181060131 
•182010131 
•182020131 
•271010131 
•?R3010131 
•283020131 
• ^283030131 
•284010131 
•4O401O131 
-•406010133 
•410010131 
•410020131 
•411010131 
•413010133 
•42101O13? 
•421020133 
•422010133 
•441010131 
•461010131 
' ♦63010131 
ERIO71010133 



i ooo) 



MAP I ON -] 
BRUSH — 
POWLER - 
Mf-An 

CONNORS 

BEPr, _ 
MI N< I Ewi cz -rvijo oq 

WALL 
KRALL 
LAYMAN 
OESILVA 
HILL 

OTjALLAGHER 
KORENMAN 
MCnONALD* F 
CHANr, — 



2201 SHOUU BE 0201 




r,LOEC<LFR — 
COPLAN 

BHA6AT — 
PRANCiF — 
YODH 
FIVEL 
BAROASISH 
BLUM . 
YOUNG 
GOLOBERG 
HOLMGREN 
PUGH 
FALtC 
REnrSH 
ANDERSON 
B S 20RN 
BRILL- 
ROOS 

MACOONALD 
Y S KIM 
tCACSER 
ALLEY 
GLOSSER 
MYFRS 
TIBELL 
GAGE 
KUNZE 
LENCHEK 



059 
113 
007 
020 

130 
395 
247 
133 
179 
021 
122 
032 
125 
115 
017 
034 
015 
022 
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•601010135 0RA6T 



•601020131 
•602010131 
♦60*^010131 
•604020131 
•606010131 
•622010131 
♦622020131 
•623010131 
♦624010131 
•624020131 
♦686010131 

♦ 704010131 
♦•'11010131 
•731010131 

♦ 7/fl0l0l3J 
♦751010131 
•76101012: 
♦832010131 
♦844010131 
♦851010131 

♦12201064? 

♦121021691 

♦429O1016? 
♦429010163 
♦6210101 '■> 
♦62101016; 
♦62101016? 




ONpnA 
KRISHFR 

SEALL 
MYCRS 
BANERJEE 
KEHOE 

STEPHENSON 
GREENBERG ' 
FERRELL 
REISER 
DORFMAN 
MITTER 
KOCH 
HORNYAK 
SNOW 

TRIVELPIECE^ 
CLICK 
GRIFFIN 
WEBER 

WALL 

BERG 

MARTIN 
PECHACEK' 
ZORNt GU 
MARTIN 
PECHACEK 



FACULTY RkTlTATION SEC" MOT SEC 105 
FACULTY L'AE SECTION - NOT SEC 215 




OU 

OIP 

010 

035 

014 

018 

028 

018 

012 

Oil 

001 

016 

014 

014 

008 

Oil 

027 

014 

010 

007 

016 

022 

001 
003 
006 
005 
003 



Cola :(ct^5/2^ -^S.l -^r tix^^fJ^Tci^ ^^f^s^ 





CI. J 



J 



I »?85Uini73 
»285010273 
r*?R50l0373 
i »285010^73 
»?R60101 71 
P#?P60] 0271 
»365010172 
• *365ni0272 
»36'i010372 
3650101 7-^ 
»3f,'^ni 0?7^ 
»365010^73 



R! CHARD 

RICHARD 

^^ICMARD 

RI CHARD 

STEINBERG 

STFlNPFRr. 

STPTNpFRr 

STPINPFRr, 

RO^^FNPFRC 

RO^^NPP^R^- 

RISK 

91SK 

RI 

ROlKH 



0 



-»2P501017? 

1 1 »285010272 

U£ii5n *?R5ni0372 

-*285nin^72 
-»285010173 
-*28501027? 
-*285010373 
-»285010^73 
♦286010171 
♦286010271 



365010172 
36501027? 
36501 0372 
♦365010173 
365010273 
365010373 





TFAM TAUGHT 
TFAM TAUGHT 
TFAM TAUGHT 
TEAM TAUGHT 

SCRAM8LED-N0T SEC 101 /TEAM TAUGH^ 
SCRAMPLFD-MOT SFC 102 /TEAM TAUGHT 
SCRAMRLFD-NOT SFC 103 /TEAM TAUGH-^ 
SCRAMBLFD-^iOT SfC 104 /TEAM TAIGHT 



TFAM TAUGHT 
TEAM TAUGHT 
TEAM TAUGHT 
TFAM TAUGHT 
TFAM TAUGHT 
TPAM TAUGHT 



PICHARD 
RICHARD 
RI CHARD 
RICHARD 
STEINBERG 
STEINBERG 
STEINBERG 
STEINBERG 
ROSENRFPG' 



ROSENBERG 



RISK 
RISK 
RISK 
ROUSH 
R0U5H 
ROUSH 




i 



ERIC 




I; 



p c 
no c 

I '* 
01' 

r)r r. 
010 

^11 
on 

004 

or ^ 
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C,2, Specification of Aggregate Titles; also shoving "comments" where appropriate 



I Each aggregate number is shown underlined, followed on the next line by its 

^ title, which is then followed on subsequent lines by comments when appropriate 

(again underlined). The number on the extreme right on the aggregate number line 
r is the number of copies needed in multiplemania, cf. section 5.6.2, p. 5.8. 



I 



(This is only a partial listing, but is part of our standard example, cf. p. Cl.l) 



Ar-r.RrGATF 
_PHv<;irs 2 S? 
l .nr.RFGATF ?o?o, 
^'HYSTf^ lftl//?ft?/3 

AGGRFGATF 70 30, 
l.'HYSICS A 20 
I vGrRFGATF 7040, 

ALI PHYSI :S SFRVTCF COURSF*^ 
r ^r-,( RFGATF" 1000, 
j >Ll Gf^NFK XL PHYSICS COlJRSFe; 
' «r.r-RFGATF 7040 « 

?TCHARn 



4. 
14, 
4. 
1 . 



DHY*^TrS — 

j| '^ INCE TH&SF COURSES WERF T^AM TA'lGHT^ THIS DE^CPIPTIO N IS f^A SEn 0.\ A n A |._ F - I r v 
I ..Gr.PFGATF 7050, ' 1. 

PHYSICS 2 35 STEINRPRG 

f)INCE THESE COURSES^ WERE J^MA TA'JGHT* THIS DESCR I PFIO N IS RASED DN \ HAlF-VIF'"' 
'^I„l.AGGRFGAfE "is" COMPUTf^O WD" I f "nfC ESS ARY ' FORCED IN._oT5)E_R_Tp 'fORM { irf:^ o^vJ^'-, 
" AGGRFGATF 7060, 
JPHYSICS 

UggR>^gatf 3?no, 3, 
-Physics eoi/? 

AGGREGATE 3? 10, 
!'HYSIC5 604/60ft 

Iggrfgatf ^??0, 3, 



PHYSICS 622/3 

f f.Gr;RCGATF 32-^0, 
'HYSICS 60I/?//604//60ft//672/3 FIRST YFAR GRADnATF COURSES 
AGGREGATE 3?ZtO, 

AiLL UPPER LEVEL PHYSICS GRADUATE COURSES 
I IgGRFGATE 3110, 

IIgENERAL) PHYSICAL 5CIFNCF<; COURSES 

aggregate 3120, 

p.ll undergraduate physics major and phsylcal sciences major courses 
Aggregate 3250, 
all physics graduate courses 

aGGRFGATE 3260, 

f.LL PHYSICS MAINSTREAM UND^RGRAD (INCL. G.P.S.) AND GRADUATE COURSES 

Aggregate ft0 30. 

PHYSICS 429//621 'GRADUATEt PHYSICS LA^ 
{f»» IGNORE THE MIDDL E SFT OF D ISTRIBUT IONS - WEIGH TED BY SECTIONS 
|i SINCE MARTIN AND PECHACFK ARE EACH IMPROPERLY COUNT E D A 5 2 SECTS . 
AGGREGATE 7160. 
HYSICS 365 



1?. 
2. 
26. 
21. 

'<7. 
3, 

?. 
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APPENDIX D. DATAREAD and INITPRT MATERIALS 

D.l, DATAREAD Materials needed for Program Execution 

D,l, a) DATAREAD Command and Descriptor C^rds as Inputted - A Complete Run Deck 

@RUN PTA* 205271KACSER^KACSER*2* 1000 

on, Ifi cuid ontij ij ont i^anU \miciizd caKd oai^out-^ltt 
©RUTT PTA* 20527 IK ACSER*KACSER*2M 000/ 10000 ^PtDic/ied coAd Q^timCLtzl 
ewSG ******^-**PLEASE SAVE PUNCHED OUTPUT 

§MSG *********** THANK Y0U - C KACSER - 205271KACSER 
ePASSWD ABCPEF - ABCPEF mpn,zbPjiU vouJi pa66m/id. 

§ASG* T PR0GRAMTAPE*8C9* P1361N -a,S4^g;t6 tliz tccpz [^?)361]on ^hlch ifiz pnognam am 
eriNDjA PRaCiRAMTAPE. DATAREADl ^tOKZd. 
§C0PIN,A PROGRAM .DATAREAD t'locjaXu VATAREAD md copiu It tzr^poKoMZy 
§FREE PRO GRAM TAPE J 11 OS 

EUhzn ("p/ie^med" option] 
@ASG# T PUNCHTAPE*8C9* SAVERl -i^ output iiZz U to be pat on a "SAUER" tapz 
§USE l^PUNCHTAPE / in &C9 ^omot [on givz actual tapz no.]. 

Ofl 

'li output iiZz U to bz put on a J'ASTUhlV ^liz ccuUzd 
PTVATAi notlcz m ^iMt dzlztz in ca^z it atnzady 
zxUtzd iJoJUtk (io\itznt6 ^nom a pnzviou^ ZKZcxition. 



eDELETE^C PTDATA 
§ASG* CPX PTDATA* F/1//2 50 
§USE 1# PTDATA 



OA. 

Wo caAjd^ - i^ output i^ to bz punckzd 
§XQT DATAREAD 

SUN IT 01 PUMCH - unit comond cojid ion output ^iiz 



SHHADFR 








'02 IC 


in 




0103 


COURSE 




0507 


SECTION 




0404 


PHYS-ASTRO-MISC 




0909 


PROF-TA 




1336 


INSTRUCTOR 




3973 


COMMENT 




7678 


SIZE 




80B0 


PHYSICS RESULTS 




8080 


SPRING 72 


SDATA 








% 0109 


ID 




121352 


SEQUENCE \ 


K 


121337 


SEQUENCE /" 


SFORM 




2 


« 


0909 


2 




0808 


-i 



hzadzn IV deA^cAipton 

kzadzn IP ^ub^iztd dz&cAipton/> 



} 
} 
} 



QznzKal hzadzn, ^izld do^cnipton^ 
hzadzn, 6izz dzJi^cnipton 

QznzKot hzadzn ^iztd dz^cnipton^ u^zd to input 
6pz(Ual inionmation [cot. SO i^ zmptu on thz 

nn^^nt^r^ n^H^ TlZadZKl] 



PROGRAM COmmES ON NEXT PAGE 



ERIC 



D2 



p 3.11 

$SX^RT - START command - MlMzd bu DIGUEK 6abde.ck, ^ 6&ct i.bCI 

#122150125 CRAIG ' ^/'^T^-f .''^fw 

020 12515 OJ 00001120 1211 Ol2l 2111 1121 - ivu^t (UgUzk coAd 

020 12515 0<00000000000000?00 ll?01ll - second digoXe-k cand 

110 ]>515 IJ 1120 2211 1111 2221 2021 - etc. 

110 12515 KO 00 00 112110 lllOOll 

OIGITEK SUBOECK COWTIWUES, both citgetefe CjOJidi and h&ad&u, till 
appKopfUatz qae^tionnaOit fL^AuttA e.xhcuK>te.d - C|5 Aectton 3.6C, p 3.11. 



SEND 
SDATA 

J 
K 

SFORM 
* 

SSTART 



- End command cxoid - aome^ at &nd a 6inglz qaz&tlomaOie. Aubdecfe. 



|oi09 
121352 
12131A 

0909 
0808 



10 

SEQUENCE 
SEQUENCE 
4 
4 
4 

- START command 
PIGITEK SUBVECK GOES HERE - 
h&adeA 

digltek coAdb 
hzadeJi 

digltek coAdA 

SEND 

NEXT OUESTIOWWAIRE CONTROL MV VIGITEK SUBOECK 
NEXT 



STARTING A WEW OUESTIOWWAIRE SUBPECK - 

&JbicQ. the. he.ade.fl iomat A.emainA unc/ionged, 
tke.fLe. U> no need to Ke.pe.at the. he.ade.fL and 
TV aomand and de^cfiiptoK coAdb. 
Sincz only the. data {^ofmat hai> changed, oA 
loeJU 04 the. qaeAtiomaJjie. nmbeA. iiom], 
only the. data, and ijo^m coAdi appeoA he.fLe.. 



pfie.(iU><Lty 04 afaoue» niitk $EWP. 



SSTOP 



STO? command - ThU teJiminate^ the. complete. OATAREAO Ve.ck. 



•FIN - UOS t&nmination ,.^„.,t »»****••******#»♦##♦# 

iMSG ##*♦**** PLEASE SAVE PUNCHED OUTPUT ♦♦#########*#***♦**** 

•MSG MANY THANKS 
•FIN 



IMPORTANT MOTES 

1. ThfLoaghout the. ai Jve., UatLc leJUeM indicate. inteApneXatcve. comenti, etc. 
and do not {^om pajvt oi the. OATAREAO decfe. Some^e^ hou)e.veA the.y do mdA,cate. 
the. pA.e4ence 0(5 IBM cxuid&. 

2. The ' at ^e be.ginning of, Aome lineA indicates a 7/S punch {i.e.. an @) 

3. Vtntical mjJUn.g& and ^maU OAXdUid numfae/ti indicaXz cxflmn^ on tkz IBM ca/id4. 



D.l.b) DATAREAD Command and Descriptor Cards as "Directly" Printed Out , with 



only minor format changes 



BXQT Question 

/i/UNiT / 1/ 0/ «/punch 
/$/HEai)ER/ 0/ 0/ 0/ 

/*/ / 0/ 2/10/lD 

/ / / 0/ 1/ 3/COuRSE 

/ / / 0/ 5/ 7/SEcTIOm 

/ / / 0/ 4/ ^/PHyS-Ac;tKO-MISC 

/ / / 0/ 9/ 9/PnoF-TA 

/*/ / O/13/06/INsTRUcTOR 

/*/ / 0/39/73/COmMENt 

/*/ / 0/76/78/SI2E 

/*/ / 0/80/80/PHySICs results 

/*/ , / 0/80/aO/SPRlNG 72 

/$/DAta / 0/ 0/ 0/ 

/ / / 0/ 1/ 9/lD 

/J/ /12/13/52/SEqUENcE 

/K/ /12/13/37/SEqUENcE 

/5/FOrm / 0/ 0/ 0/2 

/*/ / 0/ 9/ 9/2 

/ / / 0/ 8/ 8/2 

/$/STarT / 0/ 0/ «/ 



D.l.c) DATAREAD Command and Descriptor Cards as Interpreted by the Program 



questionnaire 2 
headek col. To col. 



2 10 ID 

76 78 SIZE 

9 9 2 

13 36 INSTRUCTOR 

39 73 COMMENT 

80 80 PHYSICS RESULTS 

80 80 SPRING 72 



HeadeA cand ^onmat, ci p. 3.9. 



quutlomcuAe. 2, Zoavte.d withiyi IV 



A "-t/u-cfe" aied to -input tke^e. into 
e.ach 6e.cXlon {cjoZ. SO l& blank'. ) 



DATA COL. To COL. 



1 
2 



ID 



1 

8 



9 ID 
8 2 



CARD COL. TO COL. 



13 
13 



52 DATA 
37 DATA 



COL, SEQUENCE 

12 J 
12 K 



COL. TO COL. 



1 
5 

|^ 
9 



3 COURSE 

7 SECTION 

H PHYS-ASTRO-MISC 

9 PROF-TA 



VIGITEK cxuid joHmat, c.^ 4ect i 5,p3.7 



quQJiitionnoUM. 2 ,Zocatzd imMiin IV 



iin&t data cxvvd ha& J in col 12, and 
data in col^ 13-52, second hoi K 
in cjol 12, and data in coU 13-S7. 

IP fjOmat, C|{ Tablz 3.2, p 3.4. 



Sample DATAREAD Printout for Qn^ Section 



D.2,a) Part of the Beginning of a Section Printout 



» 
» 

u 



» 
» 
» 

u 

Ul 
X 

u 



» 
» 
» 

u 

Ul 
X 

u 



» 
• 

u 

l4l 

X 

o 



» 
» 
» 

X 

u 



U 
Ul 
X 



1 




» 
» 



» 
» 
» 



» 
» 
» 



» 
» 
» 




II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

M 

If (T 
M Ul 

H a 

M < 



II 
II 
II 
II 
II 
II 
II 
II 
II 
II 
II 
II 
II 
II 
II 
II 
II 
II 
II 
II 
II 
II 
II 
II 
II 

H 
II 
II 
II 
II 
II 
II 
II 
II 
II 
II 
II 
II 
II 
II 
II 
II 
II 
II 
II 
II 

II 01 
II 

II ui 

II XtT) 

II M <U 

IK -4 
II Zo 
II2 1A 
II O ^ 
ii ^ tM 
II Kcv 

II 

M hi 
M DO 
M 





N 11 






II 


€4 




II 






II 


«4 




II 






ii 


mi 




ii 


«4 




ii 


•4 




II 


CM N 




II 




N 


II 






II 






ii 


O 01 




ii 






ii 






ii 






ii 


(MO 




ii 


«-IO 




ii 


(M 




ii 


OO 




ii 


(W o 




ii 






ii 


^ o 




ii 


o o 




ii 


o o 




ii 


OO 




II 


o o 




II 


o 




II 


o 


O 


II 


.o 


II 


o 




II 


o 


< 


II 


o 




II 




o 


II 


< < 




(g 
o 
eg 



eg 



SIS 



OOl 

cg«4 

o 

o 



o 
o 



<< 

OO 



UUI 
OO 

zz 

UUI 
OJ 30 

m 99 
iH uj Ui 

CM 

Ul 

i/> O «HM 

< ^ OO 

< o << 

O OO 



ig 



(g 

OIN 



>oi 



OlO 
OlO 
01 
0«H 



O 
OO 
OO 
OO 

o 
o 



<< 

<< 
OO 



->3£ 

UlUI 
OO 



CM 
Al 



_ UlUI 

in 03 
iH UiUI 

in 

At 

Ul 

V)0 
<M 

< o OO 

<o << 

0»4 ou 



0(M 
04t 



•04 



O^ 



OO 
OO 
OO 



< < 



<< 

OO 



uu 

OO 

zz 

UlUJ 

lo o e 

«4UIUI 

Ul 

VO O «4 M 

< ooo 
^ xcc 
<a<< 
o »^ uu 



o ^ 

C m4 
O «4 

OO 

0(g 
(g«4 

01 

|Si 

0«4 



o 
o 

is 

OO 

UlUI 

UJUJ 
lO 3D 

m o 9 

^UIUI 
Ol 

Ul 

vo o 
<ooo 

< o<< 
o •^uo 



6 



01 



01 N 



o ^ 
O CM 

0 CM 

fO o 

01 ^ 

CSJ 

o o 

^ 01 
O -4 



o o 
o o 

OO 



< < 



< < 

OO 



UJ UJ 

u o 

z z 

UlU 
D D 

o o 

Ui UJ 

(/)(/) 



Ul 

WO^ 'M 

< ooo 

< o < < 
O M o o 



1 
1 
1 
] 
] 
] 



D>2>b) The "Summed" Data Output File Printout for DATAREAD , for the Section 
shown in Appendix D.2»a. The actual output file content is also shown» 



D5 



1 1 



I V 



I i 



ERIC 



! 




< 














in 




o 






in 








1 






H 








1 U 






1 J 




t/1 


1 >^ 






1 li. 




tu 




o 




1 H 






1 3 




< 


1 Q. 


o 




1 »- 


Ui 


< 


1 D 




D 


1 O 



< O 



I 

o 

vo si 
o 



3 ^ Si o 5 



tn 
< 

I 

o 

cc 

(L 
O 

in 



o I 

o 
o cr 

o to 
< 

O I 

in 

-r 

o 
in 

O 

in ^ 



inocoininc\jino<-4 

COOCOCOOOCJ 

ooecoocooo 
oo^oooooh*o 
oo^ olO o o o oin CM 



M 
II 
tl 
II 
II 
II 
II 
It 
II 
It 
II 
II 



inooooincMinooo ii 
^ II 

OOOOOOOOOCNJO II 

II 

o oin oooooooo ii 

II 

ooo^ OOOOh*C0O II 

^ II 

ocvj ^ o o oso in^ II 
^ ^ ^11 

O CM it O lO O <M C ^ II 

^ ^ ^ II 

oooointoooo II 

^«-4 ^ fH II 

OOOOOOOOO^^ II 

II 

o oooooooo II 

II 

^Oh*OfO^OOO II 

11 



tn o 

fO UJ 

in 2 2 •-4 c^ o *>«'»4 •cNj II 

(M UJ txi ^ ^ -H II 

;tc\j2:5:o:ro^^c\joc\jind-^ ii 

CM ^ S S ^ ^ II 

oujoo^fcfinoooincMcOfO^ ii 
^inoo^^ ^ II 



^ 2^ 

6 
u 
^ in 

CM 

in ^ 



^ O O O O O O O O O II 

II 

o oooooooo II 
II 

OOOOitlOOOCMOO II 



tl 

OOO:tC0^OO K) l| 

«H II 

c c c k: ••I c rc m c ii 

CM «-4 ^ ^ «^ II 

1^ CM h*inin oin o^invo^oo ii 

CM ^ ^ ^ II 
ro ^ oooooo^oooooii 
a 2 II 

c: II 

in ^ II 

CM oooo^ ^ o ooh-m II 

CD in II 

vO CM OCMn OOCMOC^^t)0 II 

W < .J H H H M 

to CC ^ oin :f OOin Orf O O O-^ II 

•m-iuin^ II 

0<Q^UiOOsDOOOOOOOO II 

CM z o n: II 

in Z H- Or-I ooooooooo II 

sD o o in II 
DoooDoo«-ioooa^ o ii 
^ H q:*-« II 
tnK inoooin or* o oorocM ii 
inujin>- ii 
«-43ZXoo«iioor^«-4tn(MOd* li 

OM(L «^ 11 



D6 



Sample Cover Note with Distribution of INITPRT 

To AiL ?hysic3 Faculty and Teaching A35l.>eanis 
/roni: C. Macser 



Oata: July 19, 1971 



H-2rawi;:h tha first output froa the Spring 1971 teaching questionnaire. 
Ic should be self explanatory, if read with a copy of the a ppropriate ques- 
tionnaire, attached herewith. 

In a week^s tine all such outputs will be put in the Toll Lounge. If 
there is any reason to dovJbt the present output, please see ne izmediately. 
We assiised that the answer sheets in an envelope referred to the course, 
section, prof or l/A named on that envelope. We did not sort by the code 
nuaber, since it became clear that the code number, as entered, was not a 
reliable indicator. However we do have the code numbers, and it is pos- 
sible to do detective work in cases where it seems indicated. For this 
reason, we are holding back the original answer sheets for one week. In 
a week's tiae these sheets, having written coxmaents on them, will be 
available in Dr. Griggs' office for inspection there. 

Tne number of abstentions was determined from the minimm number of 
blank (f) responses, not from question 1, which was apparently answered by 
students as a question about their desires, not their acts. The individual 
counts of f were adjusted, so that the display presents f - minimum f . 
Clearly this is not quite correct. 

VJe plan to accumulate the data in reasonable ways to provide the 
departmental average distributions. E.g. sum over all sections of a course, 
S!:n over all courses of one type: Phys. 15* 16, 17, 18, etc. But this 
x^ili take time. Please be patient • 




TO: Faculty and Teaching Assi5t^t3 ror SzTi.zs 1?72 

7RCA: Dr. Bmc3 A. Barnett /^^^(^-^^ 
DATE: July 20, 1972 " * " ' 

I have reached that point in rhe analysis cf ccaching cuesticcnaire j 
for Spring 1972 vhere the instructors are asked to pcint Dcr any obvlo^zii 
errora. You should receive a conrputiir sutamary for esii ci ihe classes which 
you taught last spring. Please conxim that the caca Is corraci:, i-e. course 
nuaber^ section nunber^ enrollment » responses, etc., and if ^ ^^.rror is 
suspected contact me before August 1. 



J 

! 



APPENDIX E> SAMPLE OUTPUTS SEEN BY INDIVIDUAL INSTRUCTORS 
hli Sample PAT S Output for an Individual Section 



El,l 



u;<M 



UJ 



ERIC 



















z 








(/) 




o 


UJ . 






ce 






>• 




»- 


UJ 






> 




o 


a 










UJ 














OZ • 






zo 










• 


XK 






OU 




• 


<UJ 






















* 


•s 




• 








a > 












a 




(/) 












K 
















• 




• 


o 


O 


> 


z 








O 




UJ 






QC 




O 


M 




< 


< 




II. 


z 






z 




<A 


o 




O 












«/» 


(/) 




>• 


UJ 




X 






a 







N 



u; 



UJ 



UJ 



UJ 



tn 



UJ 



(/I 

UJ 



Ul 



Ul 



UJ 






UJ 


ct 














UJ 


< 








z 






a 


z 






z 


o 






o 








o 
















o 


hi 






Ui 




z 






o 










< 




o 


UJ 








X 






z 








UJ 




< 






o 










o 


Ct 








c 


z 


Ui 




Ul 


w 




UJ 




o 


a 


(/) 


o 






o 


UJ 




o 


X 




(/) 


u 


u 


z 








w 


U. 


o 


c 


o 


o 




z 


c* 








o 








n 


in 


< 


< 


















O 




u 


o 


z 
















UJ 








ct 




< 




UJ 












{/) 


(/I 


(/I 






H- 


H- 


Z 


Z 


Z 


Z 


UJ 


UJ 


UJ 


Ui 


u 


Q 


Q 


o 




3 








*- 






UJ 




1/) 




a 




O 


O 


o 



UJ 



UJ 



li 



♦1 



J 



F.1.2 



c 
z 



z 
z 
o 



in 



3 
o 



i4 



^0 

1 

^5 



1 
1 



a 

v> 
> 

I/* 

^•^ 

5 > 

C 



-J 

/ 

/. 



ERIC 



a 
o 

U 

rr 

2 



U 
111 



ui 

U) 

a 

O 

u 

o 
u. 

in 

-I 

in 
Ui 

hi 

a 

z 
o 

us 
o 



hi 



hi i/> 

CD I- O 
OUi 

z o 

-iH 
< tft U 

U O 
< 



2 
hi o 

O O 

< 

Z CD 
hi 

u ct: 

QC HU 
hi 

a. 1^ 
o 



oz 



OOOOOCOCCOO OCOOOCC OOCOOOOOC OGOCOC oo 

oooccoooocooococoooocoooooooooooooo 



OCCOGOOOOOOOOOOOOOOOOOOOOOOOOOOOGOO 



r 



CM 



ooooooooooooooooooooooooooooooooooo 

CN) CM *-<i lOCM WCM 

GOOOOOOOOOO 000000000^00000000003000 
lO «H K) K) lO K) CM ^ CM to lO CM CM CM * •O ^ CM 



ooooooooooocnoooooooooooooooooooooo 

CMin-l tf) CM CM K) O K) * »0 CM »0 »0 CM r4 CM -1-1 



OOOOOCC OC oo OOOOOGOCOOOOOOOOOO oooooo 



o o 



o o o o o o <> c o o 

i>C0C»OC0OOOs^O 
»4 v4 r4 



ooooooooooo o oooooo ooooooooooo oooooo 
o 

ooooooooooo ooooooo ooooooooooo oooooo 



o 



o 

CM 



ooooooooooo ooooooo ooooooooooooooooo 



o o o 
^ ^ €\i 



o 

CM 



o 

CM 



o o 



o o o 

CM CM ^ 



ooooooooooooooooooooooooooooooooooo 



o o 



O O O O O D 
CM sO O nO ^0 CM 



ooooooooo 



o o o o o 

^ X O CO ^ 



C OOCOOOOOOO OC-OC OCC'CCOOOOC. ooocoooooo 



GOO 

:t C CM 



o 
c 



o o o 
9 ^ >o 



O O G O O O C 

^ c cr ;i «o ^ 



o o o o o o 

^ CM CM CM CM 



-iCMfO ^tf>sOr*cO't:>«-tcMK)^j'>sor^<t?a^O'^tMio*;nsor*-coo* o*^c\;*Oij^io 

«H-l^«^-i«^«H«N|-i-l<MC\i<MCMCMCMCMCMCM<>f^«O«0fO*0«O 



— t 



1 

v. 

0 



] 
1 



El. 3 



1 « 



-3 



/ 2 











I- 






• 


1 


o 




z 








til 




IK 














L 


z 




o 


U 




m 














1 







I 
I 

*4 I 
I 

O I 

I 

^ I 
I 

C I 

I 
I 

Z I 

I 

O I 

I 

•X I 

I 

I- I 
I 

U I 
I 

Ui I 

I 

(/) I 
I 
I 

• I 

I 

« I 
I 

^ I 
I 

t> I 
I 

CM I 
I 

I 

UJ I 

I 

(/) I 



z 
o 
in 

Ui 

o 





Ui 






a 












< 






1 












>- 












UJ 








z 


z 


O 




o 




< 




Ui 






z 




u 




CD 


UJ 




< 


U) 


o 








o 










Ui 




X 


-i 


Ui 




a 






X 


o 


z 


o 


z 




u 


u 


o 


o 


o 


Ui 


Z 




-J 












O 


< 


< 








z 






Ui 


u 


u 








Ui 


*- 




(T 






ui 


< 


< 








(/) 






H- 


1- 




Z 


z 


z 




Ui 


Ui 


o 


a 


a 














(/) 










o 


w4 







in 



Ui 



Ui 



Ui 

GC 

< 

z 
z 
o 



o 

UI 

z 
o 

Ui 



a 
z 
o 
u 

o 

UI 

o 
oc 
z 

Ui 

o z 

tL Ui 

c 

O 3 
Ui »- 

z 

O O 

u 



o 

Z N. 

Ui 

Ui 



z 

UJ 
Q 



Z 
Ui 

u 

(T 

ui 
a 



I- 



♦ I 



» » « 

# • 

♦ # 

# < * 

• z # 

# ;^ # 
« o * 

• •^ • 

• ♦ 

♦ (A # 

# u: # 

# Z3 ♦ 

• o # 

• • 

• z « 
♦ 

• o ♦ 

* Ui * 
» ^ « 

• x: « 
#o ♦ 

• u « 

* • 

# • 

« z ♦ 

« a # 

♦ n) ♦ 

# H- ♦ 
» U) ^ 

# 4 

♦ o ♦ 
« u: ♦ 

♦ «1 ♦ 
« J » 

♦ O • 

♦ cr ♦ 
*z ♦ 

# Ui ♦ 

* # 

• * 

# z * 

# • 

« Ui * 
« a # 

# * 

# o ♦ 

♦ ♦ 
« * 
« z * 

# < * 

• X « 

♦ ♦ 

• UI * 

# ut * 

# • 

• • 

• Z # 

• z • 

• • 

* < • 

# J? » 
« * 

# • ♦ 
« • • 

• ♦ * 
4 • 

• ♦ • 



« z 

# o 

* 

• Ui 

* 

* u. 
♦ 



# * 



« • # « • • 
# • • • • 

Q 
Ui 
H- 

Z Ui 
l-Z -i 

o 

Z V 
UiU) 

< ui Ui 

Ui;/) 3 
Q< « 

O « Ui 

3 -i (/) 

XU) Z 

UJ 



3Ui 
O H' X 
:J Z 



X< it Z • 

a cz ♦ 

t/)0 o « 

J Ui Z H ♦ 

3 > Uit/) ♦ 

a i/» lij c • 

♦ «r a ; 

♦ Ui ^ Ui • 

♦ ^ 1 



• Ui 



121 



-2 



O 

o 
r 



11) 

.X 



COUHSc 1 h ^ ^ ^ SECTION OlQl 



instructor: kpall 



(ESTlV'ATE) 179 STUDENTS ViEUE ENPOLtrO IN THIS SECTION 

i22 STUDENTS PAr^TlC IPAT I ^ G Co"»*LFTEO ThE OUrsTIONNAIfcE 

1 STUDENTS PAR^ICIPATI^G ChOSE TO ABSTAIN 

56 STUDENTS V<ERE NOT ACCOUU^EO FOR 

(ESTU'ATE) 60 PE« CENT ( 122/ 179) OF sTuOENTS ENROLLFO COMPLETED ThF 0'JtSTIO.%NAlI?'=' 



♦ U A R N I N 0 : LESS THAN 70 PE?^ CFNT Or ENROLLED STUOtiNTS CO^PlETEO THE Cl'ESTIONNAIRE 



YSlCS FACUL^Yf FALL 1971 • TFACMIn6 SUKVfcY RESULTS 
IMOIVIDUAL SECTIONS 



lSTIONNAIrE no. . 2 

COURSE 4*20»* SECTION nio! 

instructor: mcoonalo* f 



ERIC 



(ESTIH^ATe) 



(ESTIMATE) 



17 STUDENTS WERE ENROLLFQ IN THIS SECTION 

0 STUDENTS PARTICIPATING Cc^-'PLETEU THE OUFST lONNAIRE 

1 STUDENTS PARTICIPATINC ChOSE TO ABSTAIN 
lo STUDENTS WERE NOT ACC0i;;;t£O .-OK 

0 PER CENT ( 0/ 17) OF STuOtNTS ENROLLED COMPLETED THE QUESTIONNAIRE 



THERF are NC RrSutTS FOrt THIS SECTION* 



£2.1 



lOlM 
&0 



E>2> Sample PATS Output for an Aggregate (Showing all 
Three Posalble Wfelgitings) 

(p.E 2.3 shows a nore conplicated aggregate having many components) 



i 

Ul 

QC 

>• 
U 
> 

S 

z 
u 

< U) 
UlUI 



«40 
9 



Ik 



9 
U 
< 

u 

M 

X 

r 



o 

Ul 

ac 

z 
zr 
o 



t/i 

Ul 
9 



si 



z 



I 
I 

• I 



I 

Ul I 

:!■ 



I 

Ul I 
I 

O I 
I 

O I 

t 

< 1 



z 
o 



u 

Ul 



U 
Ul 

u: 
>• 

OD 

O 
Ul 



Ul 
M 



Ut 

z 
o 



OP 



u. 
o 



Ul 
U) 



Ul 
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This shows the first "contents" page of the output for a more complicated aggregate 
notice the "forced" and "team- taught" components 
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APPENDIX SAMPLE AGGFORM COMPUTER RUN DECKS 



The following examples, taken and carefully studied in sequence and in 
conjunction with chapter 5 and Appendix C, should provide illustrations of most 
points, including tape handling, print file creation, (? SYMming (i.e. printing) 
output, etc. 

1. Throughout these examples, all material in script represents "inter- 
pretation" or explanation, and does not form part of the actual run deck. 

2. An * in column 1 represents a 7/8 punch, that is @. 

3. The format of run and password cards may change with time, and natur- 
ally must be the current version. Note the time and page estimates! 

4. Take special attention of the note on p. F 10! 

5. The term FASTRAND is used colloquially to refer to all drum- type mass 
storage on the 1108. 

Brief Summary of the Contents of this Appendix 

F.l. If one wishes to obtain quickly only a copy (or copiv's) of individual 
sections, to serve the same purpose as INITPRT, and using the DATAREAD output 
file as input, one proceeds as in F.l. The data can physically be either cards 
forming part of the run deck, or on tape, or be on an 1108 FASTRAND file. All 
three possibilities are illustrated. (As a somewhat superfluous test, we print 
"all aggregates" — ^which becomes an print out. This could be removed by 

changing the print control number to 00001. and deleting the first title card 
"AGGREGATES") . 

F.2. This shows how to take card output of DATAREAD, and put it into an 
1108 FASTRAND file. 

F.3, F.4, F.5 . These 3 examples show a sequence of 3 executions of AGGFORM, 
which implement the hierarchical tree of Appendix C. 

F.3 creates a printfile PFSECT for all individual sections (equivalent to 
INITPRT) , and also a temporary printfile for aggregates made in that first exe- 
cution. One copy of PFSECT is SYMmed out (printed) , and one copy of the tem- 
porary file is also automatically printed. 

F.4 creates another printfile PFAGGS which contains all aggregates computed 
so far (i.e. executions 1 and 2), and one copy of this is SYMmed out (printed) 
as a check. 

F.5 overw rites onto PFAGGS, so that it now contains all aggregates (since 
this is the last execution ). It also creates a printfile PFMULT for multiple- 
mania. Since this file is likely to exceed 250 tracks one cannot be sure that 
it can be kept on the 1108 FASTRAND. Hence this file is also copied onto a tape 
as a safety measure. Further, a diagnostic @ PRT,F PFMULT is used to determine 
the actual size of this file. 

F.6 shows how to obtain more printed copies of print files at any later time, 
provided that these files have been maintained . (At present this requires that 
they be assigned at least every 20 days, by simply using Q ASG,AX PTSECT for 
PTSECT, etc.) 

F.7 shows how to obtain further printed copies of a printfile which is 
stored on tape. 

When a set of runs has been completed, it is wise to save copies of all the 
printfiles on a tape. F.5 shows how to write from FASTRAND onto TAPE. One 
might use a tape named TAPE, and then use @COPY,M PFSECT, TAPE @COPY,M PFAGGS, 
TAPE Q COPY,M PFMULT, TAPE, etc. Further one might make both 7 track and 9 track 
versions for safety. Naturally one should also SAVE the output file tape^ OUT of 
the last execution, which contains all (binary) records. We do not give a 
complete description of tape handling. 



ERIC 



F.l. A First Execution Run which does not make aggregates. Used to obtain 

final printed section output prior to further executions which do make aggregates. 



•r>FLFTE»C PTSECT \ 
•ASG»CPX PTSECT»F/l//250 / 



•RUN PTA,20'j27lKACSER»KACSFR ♦2,100 - Ran ca/id 

•PASSWD A8CPEF - ABCVEF upfiUentA paiMCcfid 

• SyM prints* ♦PR2 - tAxck to gtttkU pfbottoat on ptintct 

•MSG THIS WILL SYM ABOUT InOO PAGES ON PR 2 ^ 

•MSG PLEAS'! CHECK TOP AND BOTTOM MARGINS AND RIBBON > - a. 

I MSG »»»»»»«»»»» THANK YOU - C KACSER - 205271KACSER J Coaxt(>.htj 

•ASG»AX KACSER*PATSY. - obtoiyUng tka pnogfum iUo. VKTSy, a^&mud to be on 

•USE P.»KACSER»PATSY. FASTRAWO. (c^ Section 5. T4,p. 5. 25) . KACSBR l6 the. obAumzd 

•ASG»AX PTDATA, - li output iU.(L 0^ VATAREAV, hdUd ccUdd PTVATA, l6 on FASTRAMP. 

OfL 

•ASG»T PTDATA»8C»STUl/ N -If, output (J-cCe 0(5 PATAREAP, ^le^e ca££ed PTPATA, l6 on tape STLiV. 

oji WOTHIWG -^(5 output iU-z OjJ PATAREAP ii on puncixad c.an.d& 

•USe'iUOUT^^*^*^^^^ " AGGFORMJ to be on a tape. SAVER. 

- pfiepMA-ng a FASTRAWP ^Ite PTSECT ^ofi 
tlxe" pfUnted" & ection output 
•USE 23»PTSECT 
•XQT P.AGGFORMl 

PHYSICS TEACHING ASSISTANTS* FALL 1971 - TEACHING JURVEY RESULTS - Ovtfiall title 
TAPE 

-99. 25, 1. 

2 I3» I2/13X»I1»7X»I9»6X»I3»7X» I3»15X,1 1 tTX* 1 1/ lOX tAAe ♦ 7X tSAS/S A6 ♦ A4 

SELECT 

END SELECT 

BLOCl 

20001. 1971. 3. .5 .51 .7 

AGGREGATES 

INDIVIDUAL SECTIONS - INPUT DATA 
END BLOCl 
END PASS 

•ADD PTDATA. - li VAT/'^EAV output ^Ue, called PTVATA, woa eltixefi on FASTRAWP m. tape, 
on. 

_PUWCHEP IBM CARPS, idUc/i (Uul the punched output oi PATAREAP, PATAREAP output ioa6 pandicd. 



^lut pntntf^ile title cajid [actually "empty") 
second pfUnt^ile titlecoAd 



} 



3 blank coAdA 



THIS IS mCJALl 



ALL DONE 
•FREF IN 
•FREf OUT 

•FREf PTSECT pilnti, section data on pfUnteA H, output i^antzd 

• SyM»U PTSECT. ♦♦PR2 -{repeat tkU cafid 04 many timeyb oA cople^b o(^ pnlnted section / 
•PMD - POSTimTEl\ PUMP - u&eial li execution 

* FIN EN/b goej> i/onong - but optional 



F.2. Setting Up a Data File (PTDATA) from card output of DATAREAD. 



•RUN PTA»205-01-002»KACSER,2»200 — Oei-s oersTic^v ^eou#^fecl 



•ASG»CPX PTDATA. 
•DATA»TL PTDATA. 



J. I 



•END 



K3 

F.'J. A First Execution Run with Aggregates . Tiiis and the subsequent runs 

corres pond to the heirarchical tree of Appendix C. 

•RUN PF,205271KACSER»KACSER»10,1000 - RUN coAd, note. tune, and pagu tuntti 
•PASSWD A8CPEF - A8CPEF fiepfLUen-ti ptUMOo^d 

«SYM PR I NTS I ♦ PR? - tficdi to get tficj pfUntoixt on pfUnten 2 

•MSG THIS WILL SYM ABOUT loOO PAGES ON PR 2 
•MSG PLEASE CHECK TOP AND BOTTOM MARGINS AND RIBBON a. uAe^ul COuHtUiJ 

•MSG »»»»»*»♦*** THANK YOU - C KACSER - 205271KACSER ^^pn 

•ASGtT OUT tSC* SAVER - output ^Ue 0^ AGGFORMI to be on SMEW 

•USE lltOUT 



•ASGtAX KACSER*PATSY. - obtaining the. pnogfuoin ^fiom FASTRAWP, cij App. F.I. 
•USE P. ♦KACSER»PATSY, 

eltlien. 

•ASG.AX PFD^TA - output ilte o(^ VKTAREAV, heAe caUed PFPATA, U on FASTRAAjP 

— 

•ASG.T PFDAi A.8C»SrUl/ N - if^ out.put ills, oi PATAREAP on tape STW. 

21 ^OTHJMG - oatpat ^ite oi VATAREAV U on punched caAdi>. 

•DFLFTE.C PFSECT - p^zpaUnq a FASTRAMP ^ile PFSECT ion. 

•ASG.CPX PFSFCT.r/l//250 tli<L " pninted " section output. 

•USE 23,PFS£CT, 

•XOT P.AGGFORMl 

PHYSICS FACULTY* FALL 1971 " TEACHING SURVEY RESUuTS - OveKoJU. titie 
TAPE 



ERIC 





-99. 


25. 




1. 


f 


213. I2/13X 


♦I1.7X.I9.6X.I3 


.7X. 13 .15 


1 


SFLECT 
AND ITEST 








1 

1 


INTERVAL 


1008. 






AND ?rEST 








INTERVAL 
AND STEST 


1008. 








INTERVAL 


-4. 


7. 


7. 




AND 3TEST 










INTERVAL 


-6. 


285. 


285. 


f • 


AND 3TEST 










INTERVAL 


-7. 


101. 


101. 




AND 4TEST 


7010. 








I NTERVAL 
AND 4TEST 


-A. 


7. 


7, 




INTERVAL 


-6. 


285. 


285* 




AND 4TEST 








f 


INTERVAL 


-7. 


102. 


10?. 




AND 5TEST 


7020. 








INTERVAL 


-A. 


7. 


7. 




AND 5TEST 








. i 


INTERVAL 


-6. 


285. 


285. 




AND 5TEST 








\ 

i 


INTERVAL 


-7. 


103. 


103. 


AND 6TEST 








INTERVAL 
AND 7TEST 


-6. 


181. 


181. 


i 


INTERVAL 


-6. 


181. 


18?. 


\ 1 


AND 8TEST 








1 


INTERVAL 


-6. 


283. 


283. 




AND 9TEST 








i ' 


INTERVAL 


-6. 


283* 


284. 




AND lOTEST 








i 


TEST 


7. 








TEST 


9« 








AND IITEST 










INTERVAL 


-6. 


410. 


411. 



MARTIN 
PECHAC 



PROGRAM COhtTWUES ON NEXT PAGE 



AND 12TFST 

INTEf<VAL -ft. 421. 42?. 

ANn nTF«;T 
tp«;t 11, 
TEST 12. 
AND 14TEST 

INTERVAL -4. 3. 3. 

AND 34TEST 

INTERVAL -6. 405. 499. 

AND 15TrST 
TEST 14, 
AND I'jTEST 
NOT TEST 13. 
AND 16TEST 
TEST 10, 
TEST 14, 
. AND 17TEST TO-^O, 

INTERVAL -4, 7. 7. 

AND 17TEST 

INTERVAL -6, 285. 285. 

AND 17TEST 

INTERVAL -7. 104, 104. 

AND 18TEST 7100. 
INTERVAL -4. 7, 7. 

AND 18TFST 

INTERVAL -6. 365. 36S. 

AND 18TFST 

INTERVAL -7. 101. lOl, 

AND 19TEST 7110, 

INTERVAL -A, 7. 7. 

AND 19TEST 

INTERVAL -6. 365. 365. 

AND 19TEST 

INTERVAL 102, 10?, 

AND 20TEST 7120, 
INTERVAL -4, 7. 7, 

AND POTEST 

INTERVAL -6. 365. 365. 

AND 20TEST 

INTERVAL -7, 103, 103, 

END SELECT 
BLOCl 

20001, 1971. 3, ,5 ,51 .7 

I AGGREGATES - lOu,t pfUtvtiUt Uttt CjCVid " 

1 INDIVIDUAL SECTIONS - 6Q.cond puintiitz tUlz cand 

\ AGGREGATE 6010, 4, . 1, 2, 1. 

I PHYSICS 429/621 - MARTIN 

I THIS APPREXRS AS 2 SECTIONS* BUT IT IS REALLY ONE SECTION. 

i THE UNDERGADUATES AND GRADUATES ARE COUNTED AS S5PARATE SECTIONS, 

I AGGREGATE 6020, 4, 2. 2, 1, 

I PHYSICS 429/621 - PECHACfK 

THIS APPEARS AS 2 SECTIONS, BUT IT IS REALLY ONE £;CTI0N, 
THE UNDERGRADUATES AND GRADUATES ARE COUNTED AS S5 »ARATE SECTIONS 
AGGREGATE 7000, 1, 3. 3, Z, 2. 

PHYSICS 285 SEC 101 

NOTE THAT STEINBERG'S STUDENTS ARE SCRAMBLED AND DO NOT t^tCESSARlLY BELONG HERE 
THIS AGGREGATE IS COMPUTED AND IF NECESSARY FORCED IN ORDER TO FORM I ATER ONES 
AGGREGATE 7010, 1. 4, 4, 2, 2. 

PHYSICS 285 SEC 102 

NOTE THAT STEINBERG'S STUDFNTS ARE SCRAMBLED AND DO NOT NECESSARILY BELONG HERE 
THIS AGGREGATE IS COMPUTED AND IF NECESSARY FORCED IN ORDER TO FORM I ATER ONES 



I ■ " 

^agcrfgate 7020, 1. 5. 5« 2» 2* 

i physics 285 sec 103 
note that steinberg's studfnts are scrambled and do not necessarily belong htrt 
This aggregate is computed and if necessary forced in order to fokm latek ones 
aggregate 3000, 6. 6. 

f PHYSICS 181 
AGGRFGATP 3010, 7. 8. 

PHYSICS 181 '2* FIRST YEAR MAJOR SEQUENCE 

J AGGREGATE 30?0, 8, 3, 

PHYSICS 283 
AGGRITGATF "^030, 0, 4, 

PHYSICS 283 /A • SECOND YEAR MAJOR SEQUENCE 

fAGGRFGATF 3^40, 10, 12. 

PHYSICS 181/2//283/4* FIRST AND SECqND YEAR MAJOR SEQUENCE 
AGGREGATE 3050, 11, 3, 

rPHYSjCS 410/411 

f AGGRF.5ATE 3060, 12, 3, 

'PHYSICS 421 '422 

-AGGREGATE 3070, 13, . 6, 

I PHYSICS 410/1//421/2' THIRD YEAR REQUIRED MAJOR COURSES 

' AGGREGATE 30R0, 15, 6. 

PHYSICS MAJOR FOURTH YEAR fLECTIVES 
TAGGREGATE 3090, 14, 12. 

I THIRD AND FOURTH YEAR PHYSICS MAJOR 

AGGREGATE 3100, 16, 24, 

|-ALL UNDERGRAD PHYSICS MAJOR COURSES 

\ AGGREGATE 7030, 1, 17, 17, 2. 2, 

^'PHYSICS 285 SEC 104 

NOTE THAT STEINBERG'S STUDFNTS ARE SCRAMBLED AND DO NOT NECESSARILY BELONG HERE 
{'this AGGREGATE IS COMPUTED AND IF NECESSARY FORCED IN ORDER TO FORM LATER ONES 
\ AGGREGATE 7100, 1, 18, 2, 

PHYSICS 365 SECT, lOl 
rAr,r,RFGATE 7110, l, 19, 2, 

} '.Physics 365 sect, 102 

aggregate 7120, 1, 20, 2, 

.-.physics 365 sect, 103 

JfND RLOCl 
* END PASS 

dLtiitn. 

r"«ADD PFDATA. - H VkTkmkV output iU.(L, QjoJULdd VWkJk, w>a4 nL^tLfL on VASTmiV on. tap&. 

I OK 

PmCHEV IBM CARPS, u)hich OAZ the. punch&d output oi VkTknEkV, U VkTkREkV output i/ocu, punch&d. 
i -99 



3 blank cxulcU 



f ALL DONE 



FREE OUT. 
'FREE PFSECT, 

•SYMfU PFSECT »»PR2 - fL&p&at tivit> cjojid ioK exich cjopy OjJ pfvintzd 6e.ctA.on output i/oanted. 

't |PMp . POSTMORTEM PUMP - u&eJuJTZfexecutlon goe4 wnong, but optional. 



!. 

•FIN 



F>A. A Second Execution Run , following that of Appendix F,3 (hence a first execution 
of AGGF0RM2). This again corresponds to the tree of Appendix C. 



•PASSWO A6CyEF -ABCVEF KzpfLtie.nt!> pa^MMfid 

•SYM PRINTS, ♦PRZ - t;Uck tC QQJt oZl pflUVtoot On pfiUltcl 

•MSG THIS WILL SYM ABOUT lOOO PAGES ON PR 2 

•MSG PLEASE CHECK TOP AND BOTTOM MARGINS AND RIBBON 1 a t^c^a£ 

tMsG **♦*♦*•♦♦♦♦♦ THANK YOU - C KACSER - 205271k;aCSER J 
•ASG^AX KACSFR»PATSY, 
•USE P.,KACSFR»PATSY. 
•ASG^T IN,8C,9553N 
♦USE 9,IN 

•ASG^T 0UT,8C»9363R 
•USE 11 ♦OUT 
•HFLt^TPtC PFAGGS 
•ASGtCPX PF^GGS♦F/l//250 
•USE 15»PFAGr,S 
•XOT P.AGGFORM? 
PHYSICS FACULTY* 



KACSER - 205271K.ACSERJ COtW,te.J// 

all pfiogfLcmi aA&' pfLZi&ntlij' in tlva ilCc. 
[cL App. F.I) 

Output i^<i- tape. 0|5 AGGFORAU becomei Input tape. ("5553) 

Output iile oi kGGTOmi put onto tape (#5363, on uic 5AI/ER) 

- pfitpoAlnQ a FASTRAMP iiZe PFAGGS ^ofi 
the " pnlnted" aggfve.gaXe outpu,t. 



ERIC 



TAPE 








1971. 


3. 






SELECT 








AND ITEST 








INTFRVAL 


-6. 


121 . 


122. 


ANn ITFST 








INTERVAL 


-A. 


2* 


2. 


AND 2. TEST 








INTERVAL 


-'>. 


2ft2. 


262. 


AND 2 TEST 








INTERVAL 


-6. 


161. 


161. 


I NTERVAL 


-6. 


262. 


263. 


AND 4TFST 








INTFRVAL 


-6. 


420. 


420. 


AND STEST 








I NTERVAL 


-4. 


2. 


2* 


AMD 6TFST 


7090. 






INTERVAL 


-6. 


286. 


286. 


AN'D 7TEST 


7040. 






I NTERVAL 


lOOP, 






AND 7TEST 








I NTERVAL 


-4. 


7. 


7. 


AND 8 TEST 


7050. 






I NTERVAL 


1008. 






AND ''TEST 


7060, 






INTERVAL 


-25. 


7000. 


7030. 


AND 10TFST 


3200. 






INTERVAL 


-6. 


601. 


602. 


AND 1 ITEST 


3210. 






INTERVAL 


-6. 


604. 


606. 


AND 12TFST 


3?20. 






I NTERVAL 


-6. 


622. 


623. 


AND 13TEST 


3230. 






TEST 


10. 






TEST 


11. 






TEST 


12. 






AND 14TEST 


3240. 






INTERVAL 


-6. 


624. 


898. 


AND 15TFST 


3110. 






INTERVAL 


-6. 


271. 


271, 


INTFRVAL 


-6. 


404. 


404. 


AND 16TEST 


3120. 







FALL 1971 - TEACHING SURVEY RESULTS - Cvefiall title. 



RICHAR 



STEINB, 



PROGRAM COmmES ON NEXT PAGE. 



I 

I 



T 



3* 

100, 
0. 



TNTEPVAL 
AND 16TF5T 
lNTFf\/AL -6. 
ANn :6TFST 
INTERVAL -25, 
AND 17TFST -^2^0, 
TFST 13, 
TF<^T 14, 
ANP 18TF«iT ^260, 
TFST 16, 
TEST 17, 
AND 19TFST 6030. 
INTERVAL -25, 
INTFRVAL lOOR, 
AND 19TF5T 
INTERVAL -4, 
AND 20TFST 7160, 
INTERVAL -25, 
FNn SFLFrT 
BLOCl 
20100, 

FXFCUTION 2 NEW AGGREOATFS 



F7 



3. 

498. 
0. 



6010. 
6. 

7100. 



6020. 
6. 

7120. 
1971. 



20RN( 



3. 



.5 



,51 



- itlnst pfUnt^Hz title, ccuid 



EXECUTION 2 - ALL AG6S SO FAR - second pfUntiUe. tUlz CJWi 

AGGPFGATF 2000. 
PHYSICS 121/2 
AGGREGATE 2010. 
PHYSIC^ 262 



1. 
2* 
3. 
4. 
5. 
6. 
7. 



AGGRFGATE 2020. 
PHYSICS 161//262/3 
AGGREGATE 2030. 
PHYSICS 420 
AGGREGATE 2040. 
ALL PHYSICS SERVICE COlJRSFS 
AGGRPGATF 7090. 4. 

PHYSICS 2P6 
AGGRFGATE 7040, 4, 
PHYSICS 285 ~ RICHARD 
SINCE THESE COURSES WERE TcAM TAUGHT. THIS DESCRIPTION IS BAbED ON A HAlF-VIEW 
AGGREGATE 7')50, 4, 8, 8, 2, 1, 

PHYS]CS 285 STEINBERG 

SINCE THESE COURSES WERE T^AM TAUGHT. THIS DESCRIPTION IS BASED ON A HAlF-VIEW 
THIS AGGREGATE IS COMPUTED AND IF NECESSARY FORCED IN ORDER TO FORM LATER ONES 



1. 



3. 
3. 
5, 
4« 
14, 
!• 
1* 



AGGREGATE 7060, 
PHYSICS 285 
AGGREGATE ^200, 
PHYSICS 601/2 
AGGRFGATE 3210, 
PHYSICS 604/606 
AGGRFGATE 3220, 
PHYSICS 622/3 
AGGREGATE 3230, 



9, 

10, 

n. 

12, 
13, 



PHYSICS 601/2//604//606//622/3 FIRST YEAR GRADUATE COURSES 

AGGREGATE 3240, 14, 

ALL UPPER LEVEL PHYSICS GRADUATE COURSES 

AGGREGATE 3110, 15, 

(GENERAL) PHYSICAL SCIENCES COURSES 

AGGREGATE 3120, 16, 

ALL UNDERGRADUATE PHYSICS MAJOR AND PHSYICAL SCIENCES MAJOR COURSES 
AGGRPGATF 3250, 17, 
ALL PHYSICS GRADUATE COURSES 

AGGRFGATE 3260. 18. PROGRAM COHTmES OH MEXT PAGE. 



2, 

3« 

3. 

3. 

9. 

12. 

2. 

26. 

21. 

47, 



ALL PHYSICS MAINSTREAM UNDfRGRAD (INCL. G.P.S.* ANu ORAUO'ATL COURotb 
AGGRPnATE eO-'^O, 19, 2. 3. 

PHYSICS A29//6?! •GRAD'JATft PhYStCS LAB 

»»»IGNORE THE MIDDLE SET OF DISTRIBUTIONS - WEIGHTED BY SECTION-^ «** 
SINLE MARTIN AND PECHAC^K ARE ^ACH IMPROPERLY 30UNTEI) AS 2 SECTS. 
ARr.PFGATF 7160, ?0, 2, 

PHV«,ir«l 36«j 
FND BLOCl 
FNO PASS 
ALL rONF 
»FREE IN, 
•FREF. OUT, 
•FREE PFAGGS 

•SYM,0 PFAG'iS. ♦ ♦PR2 - u}iJU pfiint one. copij o^"aZl"aggn.e.gat(i^, but iinco. one mo^e. excc- 
•PMD - optional ution^^tilt needed, tlvU Mill not^bo. compleXzl A uiCjJ'xC c/iccfc 



10 



r.5. A Thi rd Kxcciitlon Run following that of F.A, showing how to store Multiplemania. 



> 'RUN PF»205'»7lKACSER»KACSER»12fl500 
* 'PASSWO ABCPEF 
„«SYM PRINTSt»PR2 

' 'MSG THIS WILL SYM ABOUT loOO PAqES 



ON 



- RUM QCifid 

- ABCPEF fi&pnz&e.nti> pai&ioofLd 

- thick to geX 'ail" pfUntont on pfunte.n. 2 
PR 2 



'MSG PLEASE CHECK TOP AND BOTTOM MARGINS AND RIB80N 
• MSG *»######«## THANK YOU - C KACSER - 205271.KACSER 
"'ASGtAX KACSFR»PATSY, 
'USE P.,KACSER»PATSY. 
•ASGtT IN,8C»9363N 
'USE 9»IN 

•ASGtT OUT»8C»P870R 
♦USE llfOUT 
•OELETEfC PFAGGS 
•ASGtCPX PFAGGS»F/l//250 
USE IfitPFAGGS, 
•DELETEtC PFMULT 
-ASGtCPX PFMULT»r/l//999 
•USE 13»PFMULT, 
•XOT P.AGGFoRM? 

-PHYSICS FACULTY* FALL 1971 - TEACHING SURVEY RtlSULTS 
APE 

3, 



couhti&ij 



- atl pfioQficm M.e."p^e^e.ntlij"in thli {^ilz, 
(ciJ. App. F. 7) 

Output {jila tape. 0(J pn.^vioai axecotton bccomeA Input tape. ^9363 
Ou^tput ffite 0^ tfu-i execatcon pat onto tape {^^9870, on. oie 5AI/ER) 

- pfiepaxinQ a FASTRAMP iUc. PFAGGS ion the " pfiinted! ' aggne.- 
gate output. Notice tiiat iti pneviou^ contents oAe deleted 

- pfLepaxing a FASTRANV { tempofuviy) iile PFMULT ion muJUlpte- 
moixia. Notice lt& iizel Hence, it catmot be penmajtxent. 



1971, 

.SELECT 
'j vND ITEST 
• .NTERVAL 

AND 2TEST 
("NTERVAL 
> .ND 3TFST 

INTERVAL 
-^INTERVAL 
I NTERVAL 
SxND 3^rsT 

INTERVAL 
j'.ND AIEST 
' . NTERVAL 

END SELECT 
.-'=5 LOCI 
\ '1100. 
EXECUTION 



noe, 

7160, 
1008, 
7200, 

-:!5. 
r>08, 

-'4. 

1.000, 

-4, 



7000, 
7100, 



7. 



7030, 
71?0. 



RISK 



ROUSH 



ROSENB 



1971. 



3. 



.51 



3 - NEW AGGREGATES 



I ALL AGGREGATES 
' AGGREGATE 7140, 

PHYSICS 365 - 
t jINCE THESE COURSES 
( AGGREGATE 7150, 



tfam 



PHYSICS ' 36-5 - 
'ilNCE THESE COURSES 

GGRFGATE 7200, 
PHYSICS 285/236//365 
JVGGREGATE 1000, 



4. 

RISK 
WERE 
4, 

ROUSH 
WERE TpAM 



Pnxntout title cand ion ilfu>t pnlntiile 
Blank pnlntout title, ion second pnA-ntiile 
Pnlntout tXtti. cxuid. ion thVid pnAjntiile 
I* 1 • 



TAUGHT* 
2. 



THIS DESCRIPTION 
1. 



IS BASED ON A 



{.;XLL GENERAL 
•'cND BLOCl 
END PASS 
fvLL DONE 



TAUGHT ♦ 
3. 

STRUCTURED UNDERGRAD 
4, 



THIS DESCRIPTION IS BASED ON A 



HAlF-VIEW 
1. 

HALF-VIEW 
5, 



LAB COURSES 



PHYSICS COURSES 



j ll'PMD 
I ''FREE 
O REE 

ERIC 



IN* 
OUT. 



r 10 



C^-l.-'-i-d. 



•Af^G.T MULT .SC* SAVER - a&6igyii.ig a tape. [SAVER] onto idiidi MULTIPLEMAWIA icUi be 



•COPY PFMULT«MULT 

»MARK MOLT 
•RFWIND MULT. 
•FREE PFMULT. 
»FRFF MULT 
•PRFF PFACGG, 
•SYM,U PFA<rGS«,PR2 
•SYM,U PFMULT* tPRZ 
tOPT,F PFMULT. 
• PMO - opno. .t»\ 

END 



•FTN 



copio^ FASTRANO PFMULT onto pe^moiteaC tape MULT. 
- tape handting iMtmiction 



fizptat tkih ccJid {^oK zacii copy o^ aggregate pnxn.toat imitcd*. 
fizpzat tlia> coAd Ion eac/i copy ol muttiplcmania pn^ntotit waatcd*. 
a diagnostic to ^nd out tlic lizc of^ PFMULT. IjJ tU6 tkm 251 
t^ckb, it i^ill be saved on FASTRANP, if^ monc it laitt be dastKoytd 
and hence o^xly cxi^t on the tape MULT. 



* W/ien SVMmtwa ootpot like tills ^ one can ovcwheJbn tiic C.S.C.pninteK^ so be considerate. 
i07E It is pnjc^bdbty i^ontlmhile to get atl copies ol PFMULT ^ MULTIPLEMAMIA in this exeaiUon 
nan, since otlie^wise one i^itt have to copy it of^^ tape MULT. Sot then do not make mone 
than one cop y o^ PFAGGS at the same timei See be£ou; ^on hoi^ to make laten, copies. 



F>6> Obtaining More Printed Copies of Output * (This example prints PTSECT.) 



•RUN PTAt205-0l-271tKACSERf 1*4000 . p^^n aVid, note page^U 
•MSG THIS WILL SYM ABOUT lOOO PACES ON PR 2 / o U^i 

•MSG PLEASE CHECK TOP AND BOTTOM MARGINS AND RIBBON h \ 
•MSG THANK YOU - C KACSER* 205-01-271 J 
•ASGfAX PTSECT. 
•FREF PTSECT. 

•SYMfU PTSFCT#»»PR2 • Kepeot this co/id ^oK each copy wanted [but see note above) 

•FIN 



END 



F>7, Obtaining More Copies Multipi v^mania ^ if it is too large for FASTRAND. 

CBut first check whether the file PFMULT from F5 has satisfactory status, 
as indicated by the output from PRT,F PFMULT.; in which case proceed 
as in F.6.) 



ERIC 



•RUN 

•ASG» 

•OELE 

•ASG» 

• COPY 

•FREE 

•FREE 

»SYM« 

•DELE 

•FIN 

•FIN 



PF,20s-01-n02»KAC<iFR, to«1 500 - Run cotd, note, time and pagzi'. 

T MULT»8C«xyxxN - XXXX fiepAUenti the tapz nmbzA 

TE«C PFWULT 

AX PFMULT. «F/1//9Q9 

MULT.Pt-MULT 

MULT 

PFMULT 
U PFMULT«,pR2 
TE»C PFMULT. 



fiepzat tlvu cjoHd ioK each copy wanted {but iee note above) 



APPENDIX G — SUNDRY ADMINISTRATIVE ITEMS 

APPENDIX G.l. AGGFQRM Output Distribu tion List (Phys ics) 

1. F:ysics^ Faculty 

A: All Sections, All Aggregates > All Questionnaires : 

i public display 

ii chairman of department 

iii associate chairman 

iv teaching excellence committee 
V PATS administrator 

vi departmental administrator 

B: Individual Sections plus all Aggregates Which Contain 

Thep;f~(i»e. hie r^^ rchies) , plus Appropriate Que stionnaires: 

i to the individual 

ii & 

iii to his file (two copies) 

C: All Sections within the Jurisdiction of Course C ommittees 
or Sub commit tees f plus all Aggregates Containi ng These, 
plus Appropriate Questionnaire : 

i To course committees (e.g. Undergraduate Major 
Committee) 

ii To subcommittees (e.g. Phys 181-284 sequence) 

2. Physics > T/A*s 

A: All Sections > All Aggregates > All Questionnaires ; 

i public display 

ii chairman of department 

iii associate chairman of department 

iv teaching excellence committee 
V PATS administrator 

vi departmental administrator 

vii T/A assignment committee chairman 

viii T/A records (office) 

B: Individual Sections plus all Aggregates Which Contain 

These (i.e. hierarchies), plus Appropriate Ques tionnaire: 

i to the individual T/A 

ii to his file 

C: Al l Sections within Jurisdiction of Course C ommittees 
or Subcommittees, Plus all Aggregates C ontaining These, 
plus Appropriate Questionnaires ; 

i - To course committees (e.g. Elementary Course Comnittee) 

ii To subcommittees (e.g. Phys 161, 262, 263 subcomnittee) 

D: All Sections Supervised by One Pr ofessor, plus all Aggregates 
Containing These, plus Appropriate Questionnaire ; 



i to the professor 



G2 



G.2. Cover Memo Used with AGGFORM Printout Distribution 



UNIVERSITY OF MARYLAND 
DqMurtment of Hiysics and Astronomy 

INTRAMURAL 

To: All Faculty and Teaching Assistants .Including Course Connnittee Chairman, etc. 

From: Howel Pugh, Chairman Teaching Excellence Connittee, and C. Kacser In 
Charge of Questionnaire processing. 

Subject; 

Date: November 9, 1971 

REF: Attached Sprin,. ""l questionnaire final output (but see below) 

Attached Materials ; Herewith in a packet 
1. A copy of the appropriate questionnaire. 

Copies of all single sections with which you should be concerned 

cllill 1 "JJ individual T/A aggregates a^ appropriate. 

Copies of all aggregates which Include reference to either 2 or 3 ri,ove 

is prS f^rp rlSJ?:?^:^ JJSs^^toSS^^"^^^^^^^ indl'vldua\1e«ions a warning 

If the participation f^^'L^"L ?0^ l"gatf"i1 TlZu^Jl l^^T'' i" a^regate^ 
within the aggregate falls below 51Z tk*! ! H 5 *^°'»P"^*<* the participation 

one (e.g.. 7ol) JoJtd'^iuse Joo ^^g2e' tel tfbl r^wSd"? ""mT 'f"* * 

low, probably due to the latenesrof ! be rejected [participation was in fact 

19711 Th» if^^Z^ t ZZ ^"teness of issuing the questionnaire— last full week of Snrinc» 

in\:%Tnlt^^^^^ llX'^lKr ire"slSL?iy^^5SS/e5ected 

JndiJidSjriLlu^f J^^tJons " t/^ uiJSil mT""* °' ' accepted; and 

in an aggregate w^M at^e^t st? .f^t aggregate, are again rejected completely 

obtalnJ'JJSl."Scnr! °^ ""•P^'**' information can be ^ 

I'iok^fffil;. i« ^' Grlgg's Office. Please 

3. Toll loom : All output will soon be available in the Toll Room. 

versions of the questionnaires are being prepared now. Only minor 
editing change, are .onteq>lat.d except for questionnaire 5. If you have any suggestions 
or comments, please send them in writing to H. Pugh. suBgestions 



APPENDIX H - VERY MISCELLANEOUS ITEMS CONCERNING OPERATIONAL AND 



PROGRAMMING ASPECTS OF AGGFORM 



H.l The PQR Number 

As discussed in section 3.8b, p. 3.15 (and also section 5.2.b, 
p. 5.1), there are three possible types of weighting which may arise 
when forming aggregatas. If the PQR number is left blank on the 
aggregate control card (columns 61-70, cf. section 5.10.3.b, p. 5.22) 
then AGGFORM follows one specific standard scheme in computing these 
distributions. This choice, referred to as the "default" choice, was 
chosen as being the most natural p escription; but by setting other 
values for the PQR number one can achieve other prescriptions. 

Recall that nominally the PQ & R distributions are definaa as 
follows: 

P - distribution: all included students weighted equally 
Q - distribution: all included sections weighted equally 
R - distribution: all Included components weighted equally 

There is no difficulty or choice as regards the meaning to be 
given to the P and Q definitions, but a little thought shows that the 
R distribution is not fully defined by the above. The PQR number enables 
the user to fully specify how he wants the R distribution to be computed 
if an R distribution is to be created. 

An R - distribution is created (automatically) if the aggregate 
contains more than one component*, and at least 1 of those components 
Itself contains more than one section (a "combination" counts as a single 
sectioned section 5.2c, p5.2). That component will then necessarily 
"contain" both P and Q distributions, and may or may not also contain 
an R - distribution. In either case there may be other components of 
the aggregate which are single sections (and hence only have 
P distributions); and yet other components of the aggregate which contain 
more than one section (and hence themselves contain either P and Q; or 
P,Q and R distributions). So the question becomes "which distribution 
from each coin)onent should be used to form the R distribution of the 
final aggregate?" The PQR number specifies the choices one wishes to make. 
There are three possibilities readily available in AGGFORM (and other 
ones could be programmed as well). These are given in Table H.1.1 below. 



* It is technically possible to form an aggregate which has only one 
component - but this is of no practical significance. 



H.1.2 

Table H,l,l> Values and Meanings of the PQR Number 



PQR » Distributions Component Distribution 

number Present in Particular C osen for Weighting 

of Aggregate Component Into R Distribution 



1 P P 

(=0i "blank" P and Q P 

=default value) * P,Q and R P 



2 P P 

P and Q Q 

P,Q and R Q 

3 . P P 

P and Q Q 

P, Q and R R 



*This value is assigned if the PQR-number field on the aggregate control 
card is either left blank (and read as zero), or_ entered as zero, or_ as 1. 



The primarily anticipated use of the R distribution was to compare 
instructors who had more than one section. This can be done by 
forming an aggregate of all complete descriptions (reduced type 4) 
of the instructors one wishes to compare. The R distribution then 
attempts to weight each instructor equally. Now for questions about 
the instructor's qualities it was felt that the best instructor's average 
to take was one in which each student responding was weighted equally. 
For that reason the automatic or default option for the PQR number 
(the one obtained if the user leaves the PQR field blank) is to take the 
P distribution from each component. 



RLC 



H.2.1 



H>2 Format of the AGGFORM Output Tape Records 

R.2.1. All AGGFORM output tapes consist of (binary) records in standard 
FORTRAN tape format. Each "record" is a group of words written by a single 
FORTRAN WRITE statement. Records are separated by end-of-record gaps. The 
output tapes consist of many records, the contents of which are described 
below. 

H,l.^. Arrangement of Records 

First record - the "tape identification label", discussed briefly in 
section S.lO.l.b (p. 5.14). This consists of three words. The first two are 
specified by the user on the print control card and are usually taken to be 
the year and the semester of the survey (see section 5. 10. 3. a, p. 5.20). 
The third word is always zero. Eg 1971. 3. 0. is used for Fall 1971. (This 
follows the SUMX standard form.) 

Second and Third records - the "Directory" - see Appendix H.3. These 
two records may be very long, and contain entries for each section or aggre- 
gate, including ID number, title, etc., etc., etc. Coinraent cards and lists 
of components are stored only in the directory, not in the main records (see 
below) . 

Records 4 to (N+3) - Individual section records for the N inputted single 
sections (see below). These appear in the order in which they were originally 
assembled for DATAREAD. 

Records (N+4) to (N+W-S) * Individual aggregate records for the M aggre- 
gates constructed so far (see belcw). T- ase appear in ascending order of 
aggregate number. 

(N+Mf4)thRecord - End record (SUMX standard form) 

H.2.3 Structure of Individual Section or Aggregate Records 

These records all follow one basic pattern, which has certain variations 
for the two main cases:— section or aggregate. Recall that many items apply 
equally to sections and to aggregates. Each record may consist of up to 
1000 words; currently all records are 520 words or less. Table H. 2.3.1 on 
the next page shows the contents of each word. 



1 

2 

3 
4 
5 
6 

7 

8-; 

22 

23 

24 

25 
26 
27 

28 
29 

30 

inl 



>3>1 Format of Individual Section/Aggregate Records on Output Tapes 

Content 



For section record 



No. of students enrolled 

No. of non- abstaining respond- 
ing students ("estimated") 

//of questions 

Questionnaire No. 

ID No. 

Course No. 

Section No. 

words 8-11: Instructor's name 

words 12-17: 35 character 
"comment" from section header 
card 

words 18-19: Not used 
word 20 : course No*^ 
word 21 : section No. 
Prof-T/A number (cf section 
5.7.2, p. 5.10) 

First digit of course No. 

First digit of section No. 

Zero 
Type No. 

Response^percentage ■ lOOx 
(word 2)/(word 1) , cf section 
5.2.g, p. 5.3) 

Unused 

Unused 

Unused 



For aggregate record 



Same, but summed over included 
components 

Same, but summed over included 
components 

Same 
Same 
Zero 

Same, Ijf still applicable, 
otherwise negative 

Same, ±f^ still applicable, 
otherwise negative 

Aggregate "title" 



Same, if still applicable, 
otherwise negative 

Same, if still applicable, 
otherwise negative 

Same, if still applicable, 
otherwise negative 

Aggregate number 

Same 

lOOx (No. Included Components) /(No. of 
included + No. rejected components) 



PQR Number 

// of sections aggregated into 
this record 

// of aggregates aggregated intu 
this record 



dundant storing of these two words is used in AGGFCJRM as a patch to get this 
into the directory. The coding could be altered to do this more directly. 



H.2.3 



Tab£e H. 2. 3. T cpntinazd 



Words 


Content 






ror aj^gLcj^aue recoru 


31 


number of participating stu- 
dents who abstained ("esti- 
mated") 


Same, but summed over included 
components 


32 


Course category (cf section 
5.7.3, p. 5.10) 


No. of copies required for multi- 
plemania (cf section 5.6.2»p.5.8) 


33 


Reduced type No. (But see 
section H.5.2) 


Same 


34-40 


Unused 


Same 


41-200 


Percentage distributions 


Percentage (P-) distributions, 
w'ted by students. 


201-360 


Numerical distributions (#s) 


Percentage (Q-) distributions, 
w'ted by sections 


361-520 




Percentage (R-) distributions, 
w'ted by component, _if applic- 
able; otherwise not part of record. 



H.2.4 Further explanations 

All distributions (words 41-200, 201-360, 361-520) are entered in a 
standard format as follows. The first 2 words contain data for question i, th 
second 2 words contain data for question 2, and the words 2n-l and 2n contain 
data for question n. [As presently programmed, no more than 80 questions are 
allowed on any questionnaire.] 

For each question there are six quantities a,b,c,d,e,f representing the 
number of, or the percentage of students who marked the responses a,b,c,d,e, 
or left a blank (=f) respectively. These quantities have either integer or 
percentage values. The storage procedure is the same for both cases. The 
quantity is multiplied by 10 and any remaining fractional part is dropped. 
Thus for percentages, one place to the right of the decimal will be saved. 
These newly created integers, a,3,Y»5»e»<^ are then stored in a packed format 
in the two words of the record for that question as(ax2^** + 3x2^^ -fy) 
and (6 X 2^** + e X 2^^ + (J)) respectively. Thus 872,51,12 Is stored as 
8720 X 2^^* + 510 X 2^^ + 120), and 5.1%, 1.3%, 72.4% is stored as (51 x 2^*" + 
13 X 2^^ + 724), The words for non-existent questions have the entry "0". 

Notice that the "f" value is always stored separately and not deduced as 
being the number left over. Rounding errors will of course creep in when the 
aggregate percentage distributions are computed, and will be apparent in that 
a+b+c-l-d-l-e+f will generally not be precisely 100.0. But all such 
errors are likely to be small. 
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H. 3 Format of The Directory Records 
H«3*l Overall description 

The directory was created for several reasons. First, it was difficult to 
allow for variable-length segments within the fixed-length, fixed~lo cation 
type of record most useful in SUMX. (cf. Appendix H.4 linmediately following.) 
Such segments were needed to accomodate the (variable number of) comment car<is 
and the list of components for a given aggregate. Furthermore, it was necessary 
to have some information on each aggregate available in the computer core 
during execution; the inclusion of a table of contents at the beginning of the 
printout, for example, would be impossible without such a feature. 

The directory in fact consists of 2 separate records. The first directory 
record, called DIRl, is a "map" of the second directory record DIR2. DIR2 
contains the information pertinent to each section or aggregate, while DIRl 
enables the various programs to locate this information. 

The directory records are the two records on the tape which SUMX does not 
know about, and does not expect. Thus provision must always be made somewhere 
in the userfe non-SUMX subroutines (e.g. his BLXl) , or in the subroutine TAPESR 
for moving the tape past the ^complete directory before allowing SUMX to start 
reading section aggregate records. 

We saw in Appendix H.2 (immediately preceding) that the tape has a sequence 
of (NW) records for each of the individual N sections and M aggregates 
created so far. These are arranged, in sequence, with the N sections first, 
in the order in which they were originally read in (i.e. were assembled for 
DATAREAD); followed by the M aggregates in order of ascending aggregate number . 

In what follows, we will sometimes refer to the "nth section/aggregate 
record" or simply "the nth record". By this we mean that we start counting 
with the first section record as record no. 1, and so on through the section 
-records, and into the aggregates, with the first aggregate being the (N+l)st 
record in this scheme. 

H.3.2 DIRl Record Format 

DIRl contains information about thd location within DIR2 of the entry for 
the nth section/aggregate. It is of variable length,, with a maximum of 550 words. 
Its structure is given in Table H.3.2.1 below. 
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Table H.3.2.1 DIRI Record Format 



Word 


Contents 


1 


no. of words in DIRI 


2 


no. of words in DIR2 


3 


no. of section/aggregate records on the tape, i.e. (N-H4) 


4-17 


Title for run (c.f. section 5.10, p. 5.12) 


18 


no. of section records (N) 


19 


no. of aggregate records (M) 


20-50 


not used 


51 


location in DIR2 of beginning of 1st section/aggregate entry 
(always 1) 


52 


location in DIR2 of beginning of 2nd section/aggregate entry (= 1 + no. 
of words in 1st entry) 


53 




50+n 

• • • • 


location in DIR2 of beginning of nth section /aggregate entry (= 1 + no. 
of words in all prior entries) 



H.3.3 DIR2 Record format 

This is a variable location record, with a maximum of 20,000 words. It 
contains entries, in order, for each of the (IM/i) sections /aggregates. These 
entries can be located by use of the information contained in DIRI. Each of 
the N section entries is 19 words long, and follows the format given in 
Table H.3.3. 1 below. 

The M aggregate entries have variable length, with a maximum length of 
200 words each. They follow the format given in Table H.3.3. 2 below. 

The meaning of each of the contents in tables H.3.3.1 and H. 3.3.2 should 
be reasonably self-evident. Many of the same contents also occur in Table 
H.2.3.1, where they are cross-referenced to Ch 5 for further explanation. 
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Table H.3 .3.1 Individual Section Entry Format Within DIR2 



Word 



Contents 



1 

2 

3-6 
7-12 

13-14 

15 

16 

17 

18 

19 



no. of pages on final printout for this section 
not used 

Instructor's name 

35 character "comment" from section header card 
not used ~ 
course number 
section number 

type number (not reduced type nuni)er) 
response percentage 

questionnaire number 



Table. H. 3.3.2 on nzxt pagz. 



Table H.3.3,2 Individual Aggregate Entry Format Within DIR2 
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ERIC ; 



Word 



2 

3-16 

17 

18 

19 

20 



21 

22 
23 



24 to 
23+14XNCC 



Contents 



[no. of pages on final printout for this aggregate] 
+100x[no. of copies wanted in multiplemania] 
+10000x[no. of included sections within all components within 
this aggregate, which either pass the cutoff criteria or are 
force-accepted] 
Aggregate number 
Aggregate "title" 
type number ( not reduced type) 
response percentage 
questionnaire number 

NIC = no. of included components , which either pass the cut-off 

criteria or are force-accepted 
NRJ = no. of "excluded" components (not including force-accepted 
components) 

NFA = no. of force-accepted components 

NCC E no. of "comment" cards (cf section 5.10.3.b, p. 5.22). This 
number includes both those externally inputted using comment cards, 
and also 2 comment cards internally created for any combination 
(reduced type 1). cf footnote on p. 5.22. 

The NCC comment cards (14 words each) 



[24+14>^CC] to 
[23+14XNCC+NRJ+NIC] 



A sequential list of words generated from the entry 
number n of each of the ab initio eligible components 
for the aggregate. This includes both sections and 
aggregates. For each component, the word is taken 
as "n" for accepted components (which pass the cut- 
off criteria), is taken as "-n" if it is excluded, 
and as (n + 1000) if it is force-accepted. 



H>4 Structure of the Program AGGFORM 

The program was written around a skeleton provided by SUMX, a high energy 
physics (HEP) multipurpose sorting and data handling program* This program is 
described in ' 'SUMX Version VrV ' - by Dr. R. G. Glasser, in University of Mary- 
land Department of Physics and Astronony Technical Report #763, December 1967 
(AEC-ORO-2504-116) • This report may be obtained from the High Energy Physics 
Group in the Department. 

Not all parts of HEP*SUMX are needed for PATS. Hence a separate program 
file KACSER*SUMX was made from parts of HEP*SUMX. This makes AGGTORM self- 
contained, and ensures that any subsequent changes in HEP*SUMX do not affect 
AGGFORM. 

All program elements which were not part of HEP*SUMX and which had to be 
specially written for PATS are on a separate program file KACSER*PATSY. Both 
program files contain symbolic (FORTRAN), relocatable, and absolute elements. 
The symbolic elements of PATSY are fairly well "commented" in terms which 
should make sense to the reader of this report. This is not true for SUMX. 
Fortunately most changes which one might want to make to AGGFORM will only 
involve changes in PATSY, so that with care it should be possible to modify 
AGGFORM without changing SUMX. After any such changes, each changed sj.oolic 
element must be recompiled, to form a new relocatable element. Then all the 
programs must be remapped (@MAP), to form executable (absolute) programs. To 
successfully remap, one needs both the (recompiled) relocatable elements on 
PATSY, and the contents of SUMX, i.e. both must be on FASTRAND. Once such a 
mapping is carried out, the final executable absolute program will be on 
KACSER*AGGFORM. 

It follows that, to perform executions of AGGFORM, one only needs to copy 
the absolute elements AGGFORMl and AGGF0RM2 from tape onto FASTRAND. This 
forms a minimal executable package. 

However, to allow for possible modifications to AGGFORM, one must also 
keep both KACSER*SUMX and KACSER*PATSY on tape, and ideally have a backup copy 
of the symbolics in the form of cards. 

Putting the above considerations together, one sees that the most useful 
arrangement is a single final tape » structured as follows. The final tape 
should have 3 separate files . These should be: 

1. AGGFORMl and AGGP0RM2 absolute elementr 

2. PATSY; all symbolic, and relocatable, elements (absolute elements not 
necess ary ^ o p ti onal) 

3. SUMX; all symbolic and relocatable elements. 



H>5 Other Miscellaneous Facts Concerning the Operation of AGGFORM 

H*5*l Incorporation of more Input datf> (new Input sections) during processln^^ 

Suppose for some reason that Digit ek cards have been produced for most 
sections^ but that for some sections they are not yet available* Instead of 
delaying the Initiation of the data processing while waiting for the data sets 
to be completed, one can proceed with those data sets already available and add 
In the others at a later stage* The end result of AGGFORM will still be a single 
tape with all Input sections and all aggregates In order on It* The procedure 
Is spelled out In what follows; It has not been tested* 

a) Process the available Input sections through DATAREAD and AGGF0RM1 
exactly as described In this aannual for normal data* Continue (If 
desired) with executions of AG6F0R||I2, again exactly as spelled out before, 
making any aggregates which* do not Involve the as yet missing Input sections* 

b) Whenever the new Input sections are ready, they should be run through 
DATAREAD separately (that Is, without the previous Input*) This new 
DATAREAD output Is then fed Into AGGF0RM by a special execution of AGGFORMl* 
The only difference between this execution of AGGF0RM1 and the first one 

Is that an extra Input tape Is used In this execution, namely the tape 
containing the output of the last AGGF0RM (1 or 2) execution made with the 
first (Incomplete) set of Input sections* This tape, call It INEXTRA, with 
tape number UVUX say, should be assigned as unit 12: 

@ASG,T INEXTRA, 8C,UVWXN 
@USE 12, INEXTRA 

These cards should appear before the @XQT P* AGGFORMl card* The new Input sections 
are fed In as usual for an execution of AGGF0RM1* (That Is, they are the equivalent 
of PTDATA In example F*l, or PFDATA In example F*3 In Appendix F)* It Is Important 
to note that new aggregates formed by this execution of AGGF0RM1 can only Involve 
the new input sections* AGGF0RM1 does not have the capability of forming new 
aggregates from any sections or aggregates on the old aggregate tape (iinlt 12, 
above)* The output tape from this run of AGGF0RM1 (on unit 11) will contain 
the new Input sections and any new aggregates made from them during the execution, 
merged with the old Input sections and aggregates from unit 12* The order of 
sections and aggregate on the . output tape Is the standard AGGFORM order - Input 
sections wire first. In the order In which they were read In (putting the new 
Input sections at the end of the single section block); then the aggregates 
appear, in order of aggregate number* 

It is important to realize that new input may be added at any point in the 
series of AGGFORM executions, subject to the restrictions stated above as to 
forming new aggregates during such an execution* Once this has been done, one 
then continues with further executions of AGGFv^^M2 to make additional aggregates, 
including ones that might Involve the new input sections and the old ones* 
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H*5*2 On the reduced type number entry 

The concept of reduced type nu]d)er was introduced after the programming 
of AGGFORM was complete* It was hence effected in a patchwork fashion. This 
can never (in theory) cause any error in any AGGFORM execution, but an unwary 
user of an output tape or a future programming modification might lead to error, 

Specilically, in any AGGFORM execution, when the input is read in (whether 
the DATAREAD section output for AGGFORMl, or the output tape from a previous 
execution for AGGF0RM2) the program reads the type number for each entry and 
sets the i*-:duced type number accordingly, (This is done by the FORTRAN function- 
call M0D(IABS(N),10) wliere N is the type number,) 

ThiB is done before aay testing is done by SELECT, so that tjsts set up 
to look at the reduced type number will work. During creation of aggregates, 
however, the type number may be changed; but the program does not at such a time 
change the reduced type number correspondingly. For instance, all input sections 
are internally set as complete descriptions (type number 4) when first read in. 
But wheno during an execution an aggregate is made and assigned as a complete 
description, the type number of each of its component sections is altered from 
4 to 2. The reduced type number would (erroneously) be left at 41 Thus the 
output tape' of any execution of AGGFORM may have erroneous reduced type numbers, 
but all type numbers will be correct. 

Anyone who wishes to use output tapes for his own purpose, and also make 
use of the reduced type number, must hence either 

a) First "correct" the tape by rewriting all reduced type numbers as 
determined from the correct type numbers. Then proceed to use the reduced 
type number. OR 

b) Only use the type number; £r 

c) Read the type number, but then prior to any test, compute a valid reduced 
type number by use of MOD(IABS(N) ,10) to be used as the reduced type 

number in the test. 

H.5.3 On aggregating different questionnaires together! 

Within the present philosophy of PATS, where different questionnaires apply to 
different student populations, it should be obvious that they should not be aggregated 
together. However AGGFORM does have the capability of aggregating together components 
having different questionnaires. When components from different questionnaires 
(possibly having different numbers of questions) are aggregated togethet, each question 
number from a given component is aggregated with the same question number for all 
the other components. Whenever any single component does not have such a question 
number, no aggregate distribution is made for that question. 

If one were to use this facility one would have to arrange that all question- 
naires started with the same set of questions, and then branched for questions 
applicable only to certain populations. One would also have to ignore the computed 
and printed distributions for those question numbers that arose from what were 
in fact different questions. 
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APPENDIX J, POSSIBLE FUTURE IMPROVEMEHTS FOR PATS 

The reader Is also referred to Appendix A3^ particularly A3. 3. 

J.l Improvements for DATAREAD 

J.1.1 The problem of Identifying the number of abstainers > As dis- 
cussed In section 4.4 (p. ), the program DATAREAD Is presently defi- 
cient In that It does not directly Identify data sets which correspond to 
abstaining students. Correspondingly both INITPRT and AGGFORM contain 
algorithms to estimate this number, and "correct" the printouts and sec- 
tion records. 

. 

Clearly a more satisfactory procedure would be created If the program 
DATAREAD were suitably modified. 

J^l.2 The printout of the data sets (cf. Appendix D.2.9, p. D.4). It 
might be preferable for this to show the contents of each Dlgltek card 
exactly as read, with the columns which are considered as "true" data being 
specially marked. Recall that at present only the contents of the data 
columns are shown, these columns being specified In the data control cards 
(ct section 4.1). 

J. 2 Improvements for the AGGFORM Computing Ability 

J. 2.1 The ability to combine two "sections" to form one section . As 
discussed In section. 3. 8C (p. 3.16) [also section 5.4, p. 5.6] there can 
be certain cases where one section Is In fact subdivided Into subsections 
with different ID code nunlbers, and where the Input data Is usefully origi- 
nally presented In terms of these subsections. The current example Is 
"advanced lab" - Phys 429 (undergraduate) and Physics 621 (graduate) , which 
actually meet together as one real teaching section. For such cases, a 
useful feature would be the ability to aggregate the Initial (sub-) sections 
Into one section. (This Is to be distinguished from the situation where 
one makes a combination aggregate that counts as a simple section, ct sec- 
tion 5.1. d, p. 5.2.1 

J. 2. 2 Automatic non^^conputlng of an aggregate If It contains only 
two component sections . One of the major chores of Implementing PATS each 
semester Is the conctr^^ctlon of the hierarchical tree and the corresponding 
control cards for AGGFORM. And yet what one wants to achieve does not vary 
from semester to semester. 

At least as regards faculty, where the problem of Individual complete 
descriptions does not arise, one would like a permanent hierarchical tree 
and control cards. (A few special cases could be handled separately, by 
allowing for perhaps 4 such special aggregates per execution.) This cannot 
be done at present since quite justifiably PATS as a general rule (with 
exceptions) does not form aggregates ^ich have less than three Individual 
conq)onent sections. In the Fall semester, there are perhaps 5 sections of 



Phys 181 and 2 of Phys 182; in the Spring semester these numbers change to 
perhaps 2 sections of Phys 181 and 4 of Phys 182. 



The solution is straightforward conceptually — and probably also fairly 
straightforward to program (but care would be required to follow through all 
implications within the program). It is to add an automatic reject feature 
to prevent the creation of a specified aggregate, when an automatic test on 
the number of ab initio to-be -included sections finds this number to be less 
than some overall preset cutoff value (probably taken as 3). Naturally one 
must be able to override this rejection by an appropriate instruction for an 
individual aggregate. Such a force accept and compute feature would be 
needed to enable one to compute aggregates containing less thay (say) 3 sec- 
tions, e.g. for the (General) Physical Sciences sequence* Also this auto- 
matic reject should not apply to "combinations'', nor to subsection-section 
aggregates (see immediately* above, section J. 2.1). 

With such an automatic reject feature, one would always set up control 
cards for aggregates for both Phys 181 and for Phys 182 (as well as for 
Phys 181 + 182), etc. Thus the main part of the control card deck would not 
cliange from senestor to semester, and this would greatly simplify the imple- 
mentation of AGGFORM. 

Unfortunately this feature would not be sc useful for teaching assist- 
ants. For them, by far the largest number of aggregates are complete des- 
cription aggregates for each named T/A. This matter is discussed immediately 
below. 

J. 2. 3 Automatic computation of all complete descriptions . We mention 
this simply to point out that it is probably a pipedream — and furthermore not 
even that useful. For recall that T/As are often involved in special team 
teaching arrangements, that they soiretimes have two half-assignments, and 
that frequently they lead both lab and recitation sections. The reason this 
is likely to be a pipedream is that AGGFORM is performed in a series of exe- 
cutions, and each ececution can contain no more than twenty aggregation 
operations. 

J. 2. 4 Capability to spe cify and/or alter the number of copies of each 
aggregate which are to appear in multiplemania after havin g computed the 
aggregates. One would then be able to form the new "revised" multiplemania 
in a final execution which created no new aggregates, (cf Table 5.1 #3, 
p. 5.5). This would be useful if one found one needed more copies of only 
some complete hierarchies. 



J>3 Improvements in Printout Formats Etc. 

1. On the first page of each aggregate printout, where the included 
components are listed, state the number of responding stu'snts in each 
component. 

2. In the same place, print a "warning" signal against each component 
section for which the participation ratio lies in the marginally acceptable 
range between cutoffs #1 and 1f2 (i.e. between"50%"and "70% "participation) ; 



and print a similar warning signal for each component aggregate when appro- 
priate (cf section 5.2g, p. 

3. In general attempt to reduce the ntimber of printed pages of print- 
out « By removing unnecessary blank lines » one could; 

aO condense a section printout to two pages by putting all 
the ''front matter'' (including warning) on the same page as. the 
beginning of the tabulated distribution [Recall that no question- 
naire is ever likely to have more than 70*question8t]. Further^ 
one could then remove the 4th blank page in each section printout. 
Thus for sections one would reduce the amount of paper by a factor 
of two! 

b.) Similarly for aggregates one could attempt to condense 
the blank lines in the printout. For these it wotxld not be possible 
to squeeze everything onto two pages > since the listing of included 
and rejected components would often run over (cf. Appendix E2» p. B2.3 
et seq). For this reason it might be best always to start the tabu- 
lated distributions at the top of a new page. 



J. 4 Other Possible Modifications 

Further possible changes in AGGFORM are to liq>lement the suggestions 
made in part I of this report » specifically those discussed in sections 
1*4 (p. 1.8) and 1.7 (p. 1.9). The structure of AGGFORM Is such that most 
of these modifications can fairly readily be programmed. However many of 
them are counter to the present philosophy of the initiators of AGGFORM. 
PATS is designed to provide an in depth picture of each instructor^ to- 
gether with useful comparison "norms". To get a complete picture requires 
the presentation of lots of information. Any atteiq>t at condensation » or 
emphasis on only a few questions » is, we believe, fraught with danger. 
Even for each question the replacement of ^or supplementation of ^distribu- 
tion data by means and standard deviations ^is likely to lead to careless 
use and misuse. 

None-the-less , care has been taken in writing the programs to make it 
as easy as possible to modify them. Thus for instance there is ample space 
in each record to store means and standard deviations. 



Furthermore, as presently progranmied, the upper limit on this number is 
80 questions. 



